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Executive Summary 


This project entitled, “The Application of Unmanned Aerial Systems in Surface 
Transportation”, was undertaken as part of the Massachusetts Department of Transportation 
(MassDOT) Research Program with funding from Federal Highway Administration (FHWA) 
State Planning and Research (SPR) funds. The purpose of this research project was to assist 
MassDOT in the development of a pilot program for applying Unmanned Aerial Systems 
(UAS) to address surface transportation needs in the Commonwealth. 


In order to successfully achieve the project objectives, the following six research tasks were 
carried out: 
e Task A: Development a pilot program for integrating UAS technology into bridge and 
rail inspections 
e Task B: Assessment of roadway pavement conditions with UAS 
e Task C: Evaluation of UAS application for highway speed sensing 
e Task D: Development of UAS emergency service network for use in surface 
transportation 
e Task E: Assessment of UAS situational awareness technology and counter UAS 
(CUAS) technologies to support applications in surface transportation 
e Task F: Evaluation of UAS cybersecurity threats and countermeasures in surface 
transportation 


The research tasks were designed to support MassDOT?’s interests in applying UAS 
technology to address statewide surface transportation needs. In order to provide MassDOT 
with a comprehensive understanding of the advantages, challenges, and other aspects of UAS 
implementation, the project included six research tasks. Tasks A, B, C, and D assessed the 
potential benefits of UAS, while Tasks E and F explored mitigation strategies associated with 
the implementation of new technologies. Expanding the understanding and knowledge of the 
different aspects associated with the application of UAS is critical for the successful 
implementation and seamless integration of UAS technology into existing MassDOT surface 
transportation-related operations. 
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Organization of the Final Report 


Volume I: 


The Application of Unmanned Aerial Systems In Surface Transportation. Executive 
Summary of the Final Report (This Document). 


Volume IT: 


The Application of Unmanned Aerial Systems In Surface Transportation. Individual Final 
Reports. 


Below are the six (6) Final Report UAS research tasks: 


Task A: Development a pilot program for integrating UAS technology into bridge and 
rail inspections. 

Task B: Assessment of roadway pavement conditions with UAS. 

Task C: Evaluation of UAS application for highway speed sensing. 

Task D: Development of UAS emergency service network for use in surface 
transportation. 

Task E: Assessment of UAS situational awareness technology and counter UAS 
(CUAS) technologies to support applications in surface transportation. 

Task F: Evaluation of UAS cybersecurity threats and countermeasures in surface 
transportation. 
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1.0 Introduction 


1.1 Background 


In Phase I of the MassDOT UAS research initiative, recommendations on UAS policy, 
standard operating procedures, and best practices were formulated along with a proposed set 
of UAS applications for each MassDOT division (1). The project described in this dual 
volume final report constitutes Phase II of the MassDOT UAS research initiative and is 
designed to investigate potential UAS applications to support MassDOT’s goal of developing 
a UAS pilot program to address surface transportation needs in the Commonwealth. The 
expectation is that these UAS applications have the potential to enhance the efficiency of 
agency operations while also improving safety and security. 


1.2 Research Objectives 


The objectives of this project were: 


1. To develop a pilot program for integrating UAS into current bridge and rail 
inspection programs (Task A) 

2. To explore the use of UAS for assessing roadway pavement conditions (Task B). 

3. To evaluate the applicability of UAS for highway speed sensing (Task C). 

4. To explore how UAS can assist with emergency response in surface transportation 
and develop |UAS emergency service drone network options. (Task D). 

5. To assess commercial off-the-shelf (COTS) UAS situational awareness and CUAS 
technology to support surface transportation and protect critical transportation 
infrastructure (Task E). 

6. To evaluate UAS cybersecurity threats and countermeasures in surface transportation 
(Task F). 


The UMass Research Team in collaboration with MassDOT Aeronautics Division staff and 
other MassDOT personnel carried out all tasks. 


1.3 Final Report Outline 


The final report consists of two volumes, Volume I and Volume II. Volume I is organized as 
follows: Section 2 provides a description of each task including the problem statement and 
the objectives; Section 3 reviews the results of each task; Section 4 presents major project 
conclusions and recommendations; and Section 5 provides a list of references. A more 
detailed description of the objectives, methodology, results and conclusions of each Task is 
presented in Volume II. 
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2.0 Project Description 


The following sections provide an overview of the individual problem statements and 
objectives for individual tasks. 


2.1 Task A: Development of a Pilot Program 
to Integrate UAS Technology to Bridge and 
Rail Inspections 


2.1.1 Problem Statement 


Previously used for military reconnaissance and for strike programs, UAS technology has 
increasingly been considered for a variety of civilian tasks, including infrastructure 
monitoring, precision agriculture, package delivery services, search and rescue operations, 
photography, and more. With this task, MassDOT sought to learn from current experiences 
and challenges of bridge and rail inspections and of the use of UAS, and to develop a pilot 
program to integrate UAS into MassDOT’s bridge and rail inspections practices and 
procedures for the Highway Division, Rail and Transit Division, and MBTA. 


2.1.2 Objectives 


The objectives of this research were: 

1. To conduct a literature review of standard bridge and rail inspection procedures and 
protocols carried out by state DOTs. 

2. To explore the challenges with integrating UAS data outputs into these inspection 
procedures and protocols. 

3. To develop practical procedures and protocols for MassDOT regarding the use of 
UAS with bridge and rail inspections. 

4. To pilot test the developed procedures and protocols. 


2.2 Task B: Assessment of Roadway 
Pavement Conditions with UAS 


2.2.1 Problem Statement 


Paved roads deteriorate over time due to load and environment. To capture the extent and 
severity of the deterioration, and to decide how to best distribute available funds to keep the 
roads in service, transportation agencies and municipalities conduct pavement condition 
surveys of their roadway networks, usually on an annual or biennially basis. The data 
collected during these surveys consist of details on the distresses present on the roadway 
(cracking, raveling, weathering, etc.) and the roadway profiles. The survey process can be 
expensive, labor-intensive, and time-consuming. UAS technology offers an alternative way 
for assessing pavement condition and deterioration. UAS have the potential to evaluate large 


paved areas significantly faster than existing manual methods. In 2015, UAS technology was 
utilized by an Airsight to rapidly capture imagery, and conduct what could be classified as 
semi-automated airfield pavement condition inspections (2). The semi-automated 
assessments used a combination of CAD software technology that recognized pavement 
distresses, and human analysis of stitched imagery. Airsight reported a significant reduction 
in inspection time compared to conventional methods, which resulted in significant cost 
savings (2). Due to these potential benefits, the application of UAS technology for pavement 
condition assessment warrants further investigation. 


2.2.2 Objectives 


The main objective of this research task was to understand the current and future state of 
practice regarding the use of UAS technology for pavement condition assessment. A 
synthesis was conducted concurrently with a pavement assessment field test using UAS 
collected imagery. This task focused on addressing the following major questions: 


1. How are agencies using currently available UAS technology for pavement condition 
analysis? 

2. What are the specific current state of practice UAS platform and instrumentation 
requirements to perform pavement condition analysis? 

3. What are the specific prospective UAS platform and instrumentation requirements to 
perform fully automated pavement condition analysis? 

4. What data is collected by UAS and how is the data converted into a useable format? 

5. How do traditional pavement assessment methods compare to the semi-automated 
and fully automated methods with UAS, in terms of quality of data, cost, time, and 
safety? 


2.3 Task C: Evaluation of UAS Highway 
Speed-Sensing Application 


2.3.1 Problem Statement 


UAS capability improvements provide an opportunity to revolutionize traffic data collection 
techniques. Previously, aerial studies of highway vehicle speeds were infeasible due to the 
high cost of helicopters, and studies conducted at ground level could only capture speed data 
at specific locations along roadways. MassDOT’s Procedures for Speed Zoning on State and 
Municipal Roadways states that “it would be ideal to have speed checks at an infinite number 
of locations so that the 85th percentile speed could be computed at all points” (3) In order to 
address this need, the use of UAS as a tool to collect traffic speed data on roadways through 
the space mean speed method is being proposed. 


2.3.2 Objectives 


The objectives of this research were: 


1. To conduct a field study comparing the use of UAS to traditional speed data 
collection instruments on roadways to evaluate the feasibility and usefulness of UAS 
as a traffic data collection tool. 

2. To develop a methodology defining how aerial data can be utilized in the speed limit- 
setting process. 

3. To explore additional UAS traffic data collection uses for surface transportation 
needs beginning with an exploration of origin-destination studies. 


2.4 Task D: Development of a UAS 
Emergency Service Network for Use in 
Surface Transportation 


2.4.1 Problem Statement 


One promising application of UAS is for emergency services and disaster relief for surface 
transportation impacts. At MassDOT, the Aeronautics Division is the lead coordinating 
agency for the Massachusetts Emergency Management Agency (MEMA) on air-based 
operations. Both MassDOT and MEMA are looking to develop a better understanding of 
UAS’s capabilities to obtain rapid and critical post-disaster information to support lifesaving 
and damage assessment services. 


2.4.2 Objectives 


The main objective of this research task is to conduct a literature search and detailed 
synthesis of the application of UAS in emergency response and disaster damage assessment 
services on surface transportation networks. 

This research task will provide two deliverables: 


e Deliverable D1: Technical Memo on Literature Review 
e Deliverable D2: 
a) A GIS-based hotspot map of previous incidents and natural disasters in 
Massachusetts. 
b) An initial UAS deployment network for emergency response. 
c) Python code used to compile data and create the initial UAS deployment network. 
d) A report describing the methods to identify applicable events for UAS missions, 
development of the UAS network, and the potential ways to use the UAS network 
for decision-making. 


2.5 Task E: Assessment of UAS Situational 
Awareness and CUAS Technology in Surface 
Transportation 


2.5.1 Problem Statement 


The reduction in cost and innovations for Global Positioning Systems (GPS), cameras, and 
other advanced sensor-based technologies have led to an increase in the use of UAS. The 
FAA has estimated that there could be more than 2.5 million UAS in the United States by 
2020 that require registration (3). While promising new opportunities, this rapid proliferation 
of UAS also has the potential to harm people and property. In order to ensure public safety 
and security, there is a need to identify and test available technologies capable of detecting, 
tracking, and identifying cooperating and non-cooperating UAS operating in the open air, 
above ground level, and near important transportation infrastructure. 


2.5.2 Objectives 


The objectives of this research were twofold: 


1. To expand the MassDOT Phase I UAS 2016 literature synthesis (1) to include 
technologies that are specific to other critical surface transportation infrastructure; 
and 

2. To prepare a prototype to field test selected technologies, to assess their reliability 
and operational effectiveness related to surface transportation. 


2.6 Task F: Evaluation of UAS Cyber 
Security Threats and Countermeasures in 
Surface Transportation 


2.6.1 Problem Statement 


As public and private entities expand their use of UAS for surface transportation functions, 
UAS systems and data must be kept safe from security breaches and cyberattacks. In worse 
case scenarios, seized UAS could be used to attack critical infrastructure, particularly in 
surface transportation, and sensitive UAS-collected data could be compromised. There is an 
urgent need to ensure that UAS are engineered with sufficient security to withstand and 
recover from inevitable cyberattacks. GPS spoofing, radio controller signal interference, and 
malicious software are serious threats to UAS integrity. A comprehensive system- 
engineering plan must be specified to address the spectrum of attacks and damage to which 
UAS are susceptible. In order to enable this secure system engineering strategy, a greater 
understanding is needed of the technologies capable of cyberattacks against UAS and 
corresponding defensive strategies, and of the security standards under development for UAS 
and the compliance of COTS technologies used in building UAS. 


2.6.2 Objectives 


The objectives of this research were to identify: 


1. Technologies to attack and defend UAS from cyber threats. 

2. Security standards under development relevant to UAS. 

3. Degree of compliance of a representative sample of COTS used in the construction of 
UAS to these standards such that critical infrastructure and surface transportation can 
be defended. 
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3.0 Results 


This chapter briefly presents the results of the research tasks, including the key findings from 
literature syntheses and field tests, and key points in the proposed UAS procedures and 
protocols. Further details on the research tasks and their findings are described in the detailed 
reports for each task. 


3.1 Task A: Development of a Pilot Program 
to Integrate UAS Technology into Bridge 
and Rail Inspections 


For the literature synthesis, the research team identified and reviewed 51 inspection-related 
documents, including 43 state DOT bridge inspection manuals; two other state DOT bridge- 
related inspection documents; four state rail inspection manuals; and 2 federal bridge and rail 
inspection documents. It was found that twelve state DOTs either have implemented or 
studied the integration of UASs into their highway bridge inspection processes. It was also 
found that UAS integration into highway and rail bridge inspections could provide significant 
savings of time for both DOT inspection crew and road users, as well as reduce costs and 
improve the safety of operations. On the basis of the conducted literature synthesis, the 
research team successfully demonstrated some practical procedures and protocols for UAS 
integration into bridge and rail inspections. Those newly designed practical procedures and 
protocols have been field tested and demonstrated their effectiveness. Looking ahead, one 
preferred methodology for structures inspections (bridge and rail) is to utilize UAS software 
that would allow for autonomous flights. Similar to the reasoning behind the use of autopilot 
on manned aircraft, the use of pre-programmed UAS operations would improve safety. 


3.2 Task B: Assessment of Roadway 
Pavement Condition with UAS 


The literature review results for Task B suggested that the use of UAS for pavement condition 
assessment is still in its infancy, with little experience and information available. For 
integration with UAS, photogrammetry appears to be the most popular technique, while the 
use of multi- or hyperspectral imaging is getting increasing attention. The integration of 
LiDAR (Light Detection and Ranging) remote sensing with UAS for pavement condition 
assessment has not been thoroughly explored yet, although it does show potential. 


Besides the mentioned sensing technologies, new algorithms for analyzing pavement 
condition data were also reviewed with a specific focus on deep learning. The results 
suggested that many neural network models have been developed to detect cracks from 
images, and that these models have achieved considerable success. However, most of the 
methods are designed to identify and highlight cracked regions in pavement images with a 
rectangular shape, instead of identifying cracks at the pixel level, which is an important step 
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towards calculating pavement’s Pavement Condition Index (PCI) value. To develop deep 
neural network based crack detection algorithms, it is important to have a large database of 
pavement images with cracks clearly labeled; transfer learning appears to be one effective 
way to get around this issue. 


The research team collected a number of runway pavement images using UAS from the 
Fitchburg Municipal Airport to evaluate the pavement crack detection performance of a 
MATLAB® toolbox called CrackIT and two deep learning methods. Overall, the deep 
learning methods outperformed the CrackIT toolbox. Between the two deep learning 
methods, the U-Net model has the advantage of generating crack detection results at the pixel 
level. It also performed very well on the Fitchburg Airport data given a limited training 
dataset. 


3.3 Task C: Evaluation of UAS Highway 
Speed-Sensing Application 


Using the data collected in the field, a literature review, a methodology was developed to 
understand the accuracy of the data, and how it may be useful in the speed-limit setting 
process. Two studies were completed to understand the accuracy and cost of using UAS as 
compared to traditional methods: one for volume data collection and another for speed data 
collection. For the speed-limit setting process, more detailed speed data may be required. The 
research found that the accuracy of the speed data collected with UAS is comparable with the 
traditional methods. 


Compared to traditional methods, UAS data collection method was found to have a similar 
capital cost, while UAS has the potential to significantly reduce time, and therefore the 
operational cost, associated with data collection and processing. It was found that on medium 
to high volume roadways, UAS have the potential to be more time-cost effective than 
traditional methods. 


3.4 Task D: Development of UAS Emergency 
Service Drone Network for Use in Surface 
Transportation 


A literature review was conducted to understand the public policy and administration aspects 
of UAS applications in the United States. With the help of the literature review, an algorithm 
to design a specific UAS network for emergency response was developed on a basis of 
multiple empirical geospatial datasets, including traffic incident data, natural disaster data, 
and maintenance depot locations. The features of the traffic incidents in a five-year period 
were investigated, and then a two-step method was developed to select the applicable 
incidents for UAS applications and optimally determine the UAS network parameters. The 
first step identified the applicable incidents by the severity as well as duration filters. In the 
second step, a greedy algorithm was used to determine the number of UAS response stations 
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needed and the locations of the stations, based on the given radius of the stations and the 
expected coverage. The results from the developed algorithm indicated that the UAS network 
parameters vary with the UAS station radius and the expected coverage. It was also found 
that the UAS network developed for the traffic incidents might be capable of providing 
sufficient coverage on the influence area for many natural disasters. 


3.5 Task E: Assessment of UAS Situational 
Awareness and CUAS Technology in Surface 
Transportation 


During the literature synthesis, the research team identified 49 different products from 30 
manufacturers, based in the U.S. or overseas. Consistent with the findings from Phase I, as 
well as similar reviews and field tests conducted by others, this study have found that there is 
no CUAS product that utilizes any single type of sensing technology while at the same time 
being capable to address all challenges associated with sUAS detection, tracking, and 
identification. The most promising technologies include RF signal intelligence, EO systems, 
acoustic signature techniques, and surveillance radar. Each technology has distinctive 
advantages and drawbacks related to its capabilities, reliability, and capital and operating 
costs. Hence, the research team recommends to select products that combine multiple VAS 
detection, identification and tracking technologies that would provide the most robust 
protection for critical transportation facilities. 


Based on a preliminary evaluation carried out by the research team, seven commercially 
available CUAS products have been identified for field testing. A prototype was developed 
for pilot testing of these products and assessing their performance, capabilities, and reliability 
for protecting critical surface transportation infrastructure. In order to further improve 
understanding of challenges associated with field testing of CUAS technologies, MassDOT 
personnel and the research team attended field demonstrations of UAS and CUAS 
technology organized by one of the vendors. Field-testing of the prototype was not conducted 
during this phase of the project but is proposed for future work. 


3.6 Task F. Evaluation of UAS Cyber 
Security Threats and Countermeasures in 
Surface Transportation 


The research team found that risks can be introduced at every stage of the mission and 
business process. Inflight risks are commonly the focus of attention, due to safety concerns, 
but preflight and post-flight risks are equally if not more important. Preflight risks include 
the acquisition, assembly and configuration of the UAS hardware and software as well as 
multiple web-based applications that pose both security and privacy threats. Post flight risks 
include data processing and related storage infrastructure that threaten privacy. The business 
process helps define the mission process. Therefore, business processes can serve as a 
gatekeeper to mitigate technical risk before it is introduced. Standards are necessary but not 
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sufficient. Specifically, domain specific standards often fail to recognize the shift toward 
software-enabled capabilities or prominently emphasize corresponding cybersecurity risks 
introduced by implementing such functionality in software. As a result, these standards 
regularly fall short of offering references to quantitative procedures that can enable desired 
decision support capabilities such as design for security and cyber risk mitigation. The 
research team identified the need for simple quantitative procedures to assess cyber risk, 
compare the effectiveness of alternative countermeasures, and communicate related findings 
graphically to MassDOT who must also consider the broader business context. 
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4.0 Conclusion and Recommendations 


For each project task, the UMass Research Team conducted a literature synthesis to identify 
current practices of using UAS technology in surface transportation applications, and to 
evaluate current experiences and challenges associated with the integration of UAS 
technology into current practices. 


Based on the literature synthesis, the research team concluded that UAS technology could 
serve as a useful tool in the majority of the explored applications, including bridge and rail 
inspections; roadway pavement condition evaluation; traffic speed evaluation and 
monitoring; and emergency response and assessment. The major factors that affect the 
success of the UAS integration relate to selection of the proper types of UAS platforms and 
sensors, as well as availability and ease of use of data processing software. 


It was also found that there are a number of effective methods and techniques to mitigate 
potential threats associated with implementation of UAS technology. For example, there are 
commercially available counter-UAS technologies that can provide a comprehensive 
situational awareness in the proximity of important infrastructure and sensitive transportation 
facilities. In addition, there are effective cybersecurity methods that can reduce the risks of 
malicious cyberattacks on UAS technologies used for surface transportation functions and 
other purposes. 


Based on the results of the literature syntheses and field tests, the research team developed 
practical procedures and protocols for UAS integration into MassDOT operations. 


The UMass Research Team has developed a set of recommendations to further evaluate UAS 
applications in MassDOT divisions. The recommendations include the following: 


e Develop a pilot program for further evaluation of UAS applications in highway and 
railroad bridge inspections and in railroad right-of-way maintenance inspections in 
MassDOT divisions. Conduct field tests to evaluate UAS accuracy and efficiency in 
detection of structural deficiency of bridge structures. (Task A) 

e Conduct additional research to understand how to determine PCI values from the 
pixel-level crack detection results, and then compare the PCI values with the 
assessment results of qualified engineers. Also, a programmable procedure needs to 
be established to generate reliable and consistent PCI outcomes from UAS images. 
(Task B) 

e Explore the best vehicle tracking methods using UAS to gain the most accurate 
results. This could include volume and speed studies along with the development of a 
software platform to aid with the implementation of UAS for traffic data collection. 
Other potential uses of UAS for traffic monitoring could also be investigated 
including turning movement counts, conflict-event studies, intersection delay 
measurements, parking utilization tracking, and queue studies. (Task C) 

e Conduct a pilot study to improve the network design and to optimize the location of 
UAS stations for emergency response, assessment, and management. Specifically, 
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there is a need to calibrate the operational parameters of drones and identify potential 
issues that may arise in practice. Additionally, the network design should refine the 
criteria filters to incorporate more operational characteristics of drones. (Task D) 

e Conduct a field study to evaluate selected CUAS technologies to test their detection, 
tracking, and identification capabilities in a real-world environment under a variety of 
environmental conditions and with different UAS products A special effort should be 
made to test CUAS against intruder drones flying in fully autonomous, radio-silent 
mode. (Task E) 

e Interms of cybersecurity and security risks, the two main recommendations are to 
survey the MassDOT UAS mission portfolio to identify where cyber risk assessment 
can be applied for the greatest benefit and to assess and certify humans and UAS to 
prevent and close gaps in the mission and business processes of the 
organization/agency. (Task F) 


It is recommended that future MassDOT research focus on bringing UAS technology 
applications from the state-of-the-art to the state-of-the-practice. 


A more detailed presentation of the objectives, methodology, results and conclusions of each 
Task is presented in Volume II. 
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Executive Summary 


This study entitled, Development of a Pilot Program to Integrate UAS Technology into 
Bridge and Rail Inspections, was conducted as part of the Massachusetts Department of 
Transportation (MassDOT) Research Program. This program is funded with Federal 
Highway Administration (FHWA) State Planning and Research (SPR) funds. Through this 
program, applied research is conducted on topics of importance to the Commonwealth of 
Massachusetts transportation agencies. The study included four subtasks: 


1. Conduct a literature review of standard bridge and rail inspection procedures and 
protocols carried out by state DOTs. 

2. Review the experiences and challenges associated with integrating UAS into the 
inspection procedures and protocols, and with other data collected during inspections. 

3. Develop practical procedures and protocols for MassDOT for using UAS in bridge 
and rail inspections. 

4. Test these UAS based procedures and protocols in collaboration with MassDOT staff. 


Based on the results of sub-tasks 1 and 2, the UMass research team concluded that UAS can 
serve as a useful tool for MassDOT in a majority of bridge and rail inspection procedures, 
with the exception of in-depth bridge inspections, because these types of inspections require 
hands-on testing. The major factors that affect the success of UAS integration into these 
inspections relate to selection of the proper types of UAS platforms and sensors. It is 
recommended that a rotorcraft UAS platform be used for the majority of bridge inspections 
and fixed wing platform for general surveys of extended areas, such as railroad right-of-way. 
The most useful sensors for bridge inspections include thermal sensors to detect areas of 
bridge deck delamination and high-zoom visual spectrum cameras to facilitate the close-up 
inspection of joints, bolts, and welds and to identify delineations like stress cracks in the steel 
structures, bridge decks and other elements. Also, LiDAR sensors are recommended for asset 
management to provide high definition measurements of transportation infrastructure; to 
conduct right-of-way surveys; to create 3D models; and to detect the presence of 
transportation infrastructure elements such as bridges, light poles, and signs. UAS can also be 
implemented to assist with rail bridge inspections, construction, and general maintenance and 
right-of-way inspections. However, because of the current FRA regulations, UAS cannot be 
used for the annual routine inspection of railroad tracks. 


The results of sub-task 3 and Section 4.0 included the development of practical procedures 
and protocols for MassDOT to integrate the use of UASs into future bridge and rail 
inspections. The procedures and protocols have taken into consideration the major issues, 
challenges, practices, and lessons learned from existing UAS based practices, including the 
UAS data collection, storage, and dissemination. The procedures and protocols were 
developed with MassDOT’s specific needs and organizational constraints in mind and efforts 
were made to ensure that any new procedures and checklists developed specifically for 
bridge or rail inspections conform to existing policies and procedures at MassDOT. 
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1.0 Introduction 


1.1 Problem Statement 


As aresult of Phase I of the MassDOT UAS research initiative, recommendations on UAS 
policy, standard operating procedures, and best practices were formulated along with a 
proposed set of potential UAS applications for each MassDOT division [1]. Building on the 
results of Phase I research, a research project scope (referred to as Phase II Task A) was 
developed and designed to create a draft internal policy as well as a draft standard operating 
procedure to support MassDOT’s goal of developing a UAS pilot program within the agency. 


1.2 Research Objectives 


The objectives of this research were: 1) to conduct a literature review of standard bridge and 
rail inspection procedures and protocols carried out by state DOTs; 2) to explore the 
challenges with integrating UAS data outputs into these inspection procedures and protocols; 
3) to develop practical procedures and protocols for MassDOT regarding the use of UAS 
with bridge and rail inspections, and 4) to test the developed procedures and protocols. 


These sub-tasks have been carried out by the UMass Research Team in collaboration with 
MassDOT Aeronautics Division staff and other MassDOT personnel. 


1.3 Report Outline 


The remainder of this report is organized as follows. Chapter 2 describes the research 
methodology. Chapter 3 presents the results of the literature syntheses, outlines key points of 
the developed procedures and protocols, and discusses the field testing. Chapter 4 provides 
conclusions and recommendations. Chapter 5 provides a list of references. 
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2.0 Research Methodology 


The first task of this project was to conduct a literature synthesis of procedures and protocols 
for bridge and rail inspections. A literature review was conducted to identify actions taken by 
state DOTs regarding their use of UAS to conduct inspections of bridge and rail 
infrastructure. Technical reports, journal articles, and other written documents, as well as 
Internet-based sources, were reviewed for the purposes of identifying major issues, 
challenges, practices, and lessons learned. Federal and state statutory, regulatory, and 
procedural requirements pertaining to bridge and rail inspections were also reviewed, and the 
usefulness of UAS technology in meeting these requirements was explored. 


The second task of the project was to explore challenges with integrating UAS data outputs 
into inspection procedures and protocols, and with other data collected during inspections. 
State DOTs using UAS for bridge or rail inspections were contacted to obtain input and 
feedback regarding the quality of the data collected with UAS technology compared to 
traditional inspections. The extent to which UAS data outputs are integrated into the existing 
bridge and rail inspection products and analysis tools was also reviewed. 


The third task of this project was to develop practical procedures and protocols for MassDOT 
for integrating UAS into future bridge and rail inspections. The developed procedures and 
protocols took into consideration major issues, challenges, practices, and lessons learned 
from existing practices, including regarding the collection, storage, and dissemination of 
UAS data. The procedures and protocols have been designed to take into account 
MassDOT’s specific needs and organizational constraints. Efforts have been made to ensure 
that any new procedures and protocols developed specifically for bridge or rail inspections 
conform to existing policies and procedures at MassDOT. 


The fourth task of the project was to field test the developed procedures and protocols, to 
evaluate and ensure their effectiveness. The time, location, personnel, and equipment 
involved in this testing were determined in consultation with MassDOT. 
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3.0 Results 


This chapter presents the results of the literature syntheses, outlines key points of the 
developed procedures and protocols, and discusses the field testing. 


3.1 Literature Review of Procedures and 
Protocols for Bridge and Rail Inspections 


This section presents the major findings of the literature review conducted to complete Sub- 
Task 1. The UMass research team identified and reviewed 49 documents, including 41 state 
DOT bridge inspection manuals; two other state DOT bridge-related inspection documents; 
four state rail inspection manuals; and two federal bridge and rail inspection documents. 
These documents are listed in the References in Chapter 5. 


A summary of the literature review is presented below and includes the following 
subsections: 

e An Overview of the Current Highway Bridge Inspection Procedures 

e An Overview of the Current Rail Inspection Procedures 


3.1.1 An Overview of the Current Highway Bridge Inspection Procedures 


The National Highway Bridge Inspection program was established in response to the collapse 
of the Silver Bridge over the Ohio River between West Virginia and Ohio; this tragic 
accident killed 46 people in 1967. At that time, the exact number of highway bridges in the 
United States was unknown, and there was no comprehensive bridge inspection program to 
monitor the condition of existing bridges. 


In the Federal-Aid Highway Act of 1968, Congress directed the Secretary of Transportation 
in cooperation with state highway officials to establish: 1) National Bridge Inspection 
Standards (NBIS) for the proper safety inspection of highway bridges; and 2) a program to 
train employees involved in highway bridge inspection to carry out the program. As a result, 
the NBIS regulation was developed, a highway bridge inspector’s training manual was 
prepared, and a comprehensive training course based on the manual was developed to 
provide specialized training. To address varying needs and circumstances, state and local 
standards are often even more restrictive than the national standards. 


According to the U.S. Department of Transportation, Federal Highway Administration’s 
(FHWA) Bridge Inspector’s Reference Manual (BIRM) (2), all highway bridge inspection 
procedures conducted by state DOTs are divided into seven categories: 1) initial or inventory 
inspections; 2) routine periodic inspections; 3) damage inspections; 4) in-depth inspections; 
5) fracture critical inspections; 6) underwater inspections; and 7) special interim inspections. 
A brief summary of these highway bridge inspections categories and procedures is presented 
in Table 1. 


Table 1. Summary of current U.S. highway bridge inspection procedures 


Procedure 


Description 


Initial 
(Inventory) 
Inspection 


The first inspection of a bridge when the initial inventory file is created. The elements of an initial 
inspection may also apply when there has been a change in the configuration of the structure (e.g., 
widening, lengthening, supplemental bents, etc.) or a change in bridge ownership. The initial 
inspection is a fully documented investigation and is accompanied by load capacity ratings. The 
purpose of this inspection is two-fold: 1) provide all Structure Inventory and Appraisal (SI&A) 
data, and 2) provide baseline structural conditions and identification of existing problems. 


Routine 
(Periodic) 
Inspection 


Regularly scheduled inspections consisting of observations and/or measurements needed to 
determine the physical and functional condition of the bridge, to identify any changes from “initial” 
or previously recorded conditions, and to ensure that the structure continues to satisfy present 
service conditions. Inspection of underwater portions of the substructure is limited to observations 
during low-flow periods and/or probing for signs of scour and undermining. According to the NBIS, 
standard bridge inspection intervals should not exceed 24 months. However, certain bridges require 
inspection at less than a 24-month interval. The procedure establishes criteria to determine 
inspection frequency and intensity based on such factors as age, traffic characteristics, and known 
deficiencies. Certain bridges may be inspected at greater than 24-month intervals, but not to exceed 
48 months, with prior FHWA-approval. This may be appropriate when past inspection findings and 
analysis justify the increased inspection interval. 


Damage 
Inspection 


An unscheduled inspection to assess structural damage resulting from environmental factors or 
human actions. The scope of inspection is sufficient to determine the need for emergency load 
restrictions or closure of the bridge to traffic and to assess the level of effort necessary for an 
effective repair. 


In-Depth 
Inspection 


A close-up inspection of one or more members above or below the water level to identify any 
deficiencies not readily detectable using routine inspection procedures. Hands-on inspection may be 
necessary at some locations. When appropriate or necessary to fully ascertain the existence of, or 
the extent of, any deficiencies, nondestructive field tests may need to be performed. The inspection 
may include a load rating to assess the residual capacity of the member or members, depending on 
the extent of the deterioration or damage. This type of inspection can be scheduled independently of 
a routine inspection, though generally at a longer interval, or it may be a follow-up for other 
inspection types. For small bridges, the in-depth inspection includes all critical members of the 
structure. For large and complex structures, these inspections may be scheduled separately for 
defined segments of the bridge or for designated groups of elements, connections, or details. NBIS 
establish criteria to determine the level and frequency of this type of inspection. 


Fracture 
Critical 
Inspection 


The fracture critical members (FCM) inspection uses visual methods that may be supplemented by 
nondestructive testing. A very detailed visual hands-on inspection is the primary method of 
detecting cracks. This may require that critical areas be specially cleaned prior to the inspection, and 
additional lighting and magnification be used. Where the fracture toughness of the steel is not 
documented, some tests may be necessary to determine the threat of brittle fracture at low 
temperatures. According to the NBIS, FCMs should be inspected at regular intervals not to exceed 
24 months. However, certain FCMs require inspection at less than 24-month intervals. The 
procedure establishes criteria to determine the inspection level and frequency of inspections, 
considering such factors as age, traffic characteristics, and known deficiencies. 


Underwater 
Inspection 


Inspection of the underwater portion of a bridge substructure and the surrounding channel, which 
cannot be inspected visually at low water by wading or probing, generally requiring diving or other 
appropriate procedures. Underwater inspections are an integral part of a total bridge inspection plan. 
According to the NBIS, underwater structural elements are inspected at regular intervals not to 
exceed 60 months. However, certain underwater structural elements require inspection at less than 
the 60-month intervals. Also, certain underwater structural elements may be inspected at greater 
than 60-month intervals, not to exceed 72 months, with written FHWA approval. This may be 
appropriate when past inspection findings and analysis justify the increased inspection interval. 


Special 
(Interim) 
Inspection 


Intended to monitor a particular known or suspected deficiency, such as foundation settlement or 
scour, fatigue damage, or the public’s use of a load posted bridge. These inspections are not usually 
comprehensive enough to meet NBIS requirements for routine inspections. Inspection scheduled at 
the discretion of the bridge owner. 


In order to match their specific needs and to further enhance federal highway bridge 
inspection procedures and requirements, many state DOTs developed their own highway 
bridge inspection manuals. Thirty-six state DOTs have published their highway bridge 
inspection manuals online. Seven additional state DOTs provided their highway bridge 
inspection procedure manuals via email. Another seven state DOTs responded and indicated 
that they completely follow FHWA’s BIRM procedures for highway bridge inspections. 


The states with their own highway bridge inspection manuals are: Alabama [3], Alaska [4], 
Arizona [5], Arkansas [6], California [7], Colorado [8], Connecticut [9], Delaware [10], 
Florida [11], Hawaii [12], Idaho [13], Illinois [14], Indiana [15], lowa [16], Kansas [17], 
Kentucky [18], Louisiana [19], Massachusetts [20], Michigan [21], Minnesota [22], 
Mississippi [23], Missouri [24], Montana [25], Nebraska [26], Nevada [27], New 
Hampshire [28], New Jersey [29], New York [30], North Carolina [31], North Dakota [32], 
Ohio [33], Oklahoma [34], Oregon [35], Pennsylvania [36], Rhode Island [37], Tennessee 
[38], Texas [39], Utah [40], Virginia [41], Washington [42], West Virginia [43], and 
Wisconsin [44]. However, it was found that the majority of those state manuals almost 
completely follow FHWA’s BIRM, while only 13 of those state bridge inspection manuals 
add more detail to inspections and an additional inspection type called a complex highway 
bridge inspection. The 13 states with these additional details are Florida, Hawaii, Michigan, 
Missouri, Nevada, New York, Ohio, Oregon, Rhode Island, Utah, Virginia, and Washington. 
There are seven state DOTs that do not have a written bridge inspection manual but instead 
use the BIRM as their state bridge inspection manual. Those states are Georgia, Maine, 
Maryland, New Mexico, South Carolina, South Dakota, and Wyoming. 


3.1.2 An Overview of the Current Rail Inspection Procedures 


State DOTs were contacted by the research team regarding their current procedures and 
regulations for railroad inspections. Thirteen state DOTS responded, nine of them do not 
currently have any procedures or regulations for railroad inspections but instead use Federal 
Railroad Administration (FRA) regulations and procedures when conducting such 
inspections. The nine states are Arizona, Colorado, Florida, Georgia, Louisiana, Nebraska, 
North Carolina, Ohio, and Tennessee. Based on the literature review, all rail inspections can 
be divided into two categories: 1) rail bridge inspection; and 2) rail track inspection. The two 
different types of rail inspections, along with the FRA procedures and regulations, are 
described as follows: 


3.1.1.1 Rail Track Inspection Procedures 

The main guide for conducting rail track inspections is the FRA’s Track and Rail and 
Infrastructure Integrity Compliance Manual, Vol. II [45]. This manual contains the 
minimum requirements for the frequency and manner of inspecting the track. It also states 
that the track owner may exceed the Track Safety Standards (TSS) [46] in the interest of 
good practice, but they cannot be less restrictive. For compliance with the TSS, each 
inspection shall be made on foot or by riding over the track in a vehicle at a speed that allows 
visual inspection of the track structure. Mechanical, electrical, and other track inspection 
devices may be used to supplement visual inspection. If a vehicle is used for visual 
inspection, the speed of the vehicle is limited to five miles per hour when passing over track 
crossings and turnouts; otherwise, the speed of the inspection vehicle shall be set at the sole 


discretion of the inspector, based on track conditions and inspection requirements. The 
following four requirements should be obeyed when performing an inspection while riding 
over the tracks in a vehicle [46]: 


1. One inspector in a vehicle may inspect up to two tracks at one time, provided that the 
inspector’s visibility remains unobstructed by any cause and that the second track is 
not centered more than 30 feet from the track upon which the inspector is riding. 

2. Two inspectors in a vehicle may inspect up to four tracks at a time, provided that the 
inspectors’ visibility remains unobstructed by any cause and that each track being 
inspected is centered within 39 feet from the track upon which the inspectors are 
riding. 

3. Each main track should be traversed by a vehicle or inspected on foot at least once 
every two weeks, and each siding should be traversed by the vehicle or inspected on 
foot at least once every month. On high-density commuter railroad lines, where track 
time does not permit any on-track vehicle inspection, and where track centers are 15 
feet or less, the requirements of this paragraph will not apply. 

4. Track inspection records shall indicate which track(s) are traversed by a vehicle or 
inspected on foot. 


An inspection made from a road vehicle driven alongside the track cannot substitute for 
regular inspections but can be used for supplemental purposes. The railroad operator may 
implement additional inspection procedures, provided that these inspections are only used for 
supplemental purposes. All state DOTs and track owners must follow the procedures 
described previously when completing a rail track inspection [45]. 


3.1.1.2 Rail Bridge Inspection Procedures 

According to the FRA’s Track and Rail and Infrastructure Integrity Compliance Manual, 
Vol. IV, Chapter 1, Bridge Safety Standards [47], all railroad bridge inspection procedures 
conducted by state DOTs must include the following categories: 1) Routine or periodic 
inspections; 2) seismic inspections; 3) underwater inspections, and 4) special inspections. 
The FRA has not established specific standards or procedures for inspections of railroad 
bridges but instead provides requirements such as frequency, type of inspection, and 
reporting that track owners are supposed to incorporate into their railroad bridge inspection 
procedures. A brief summary of the current types of FRA-mandated bridge inspections [46] 
is presented in Table 2. 


Table 2. Summary of current U.S. railroad bridge inspection procedures 


Inspection Description 


Periodic Intended to determine whether a structure conforms to its design or rating condition and, 
Inspection | if not, the degree of nonconformity. Section 237.101(a) calls for every railroad bridge to 
be inspected at least once each calendar year. Deterioration or damage may occur during 
the course of a year, regardless of the level of traffic that passes over a bridge. 
Inspections at more frequent intervals may be required due to the nature or condition of a 
structure or intensive traffic levels. 


Seismic Intended to reduce the risks posed by earthquakes in the areas in which their bridges are 
Inspection | located. Precautions should be taken to protect the safety of trains and the public 
following an earthquake. Contingency plans should be prepared in advance and consider 
the potential for seismic activity in an area. When a major seismic activity occurs, all 
railroad bridges within the epicenter of the earthquake shall be inspected for damage. 


Underwater | Intended to measure and record the condition of substructure support at locations subject 

Inspection | to erosion from moving water. Stream beds often are not visible to the inspector. Indirect 
measurements by sounding, probing, or any other appropriate means are necessary in 
these cases. Where such indirect measurements cannot provide the necessary assurance 
of foundation integrity, diving inspections should be performed as prescribed by a 
competent engineer. 


Special Intended for bridges that might have been damaged by a natural or accidental event, 
Inspection | including but not limited to a flood, fire, earthquake, derailment, or vehicular or vessel 
impact. Requires the track owner to have in place a means to receive notice of such an 
event, including weather conditions and earthquakes, and a procedure to conduct an 
inspection following such an event. In order for these procedures to effectively protect 
train operations, instructions should provide details on required procedures, including 
any restrictions, and must be issued to transportation personnel responsible for the 
dispatching and operations of trains. 


In addition, there are four key points that FRA requires to be incorporated into railroad 
bridge inspection procedures by railroad owners [47]: 


1. Each bridge management program shall specify the procedure to be used for the 
inspection of individual bridges or classes and types of bridges. 

2. The bridge inspection procedures shall be specified by a railroad bridge engineer, 
who is designated to be responsible for the conduct and review of the inspections. 
The inspection procedures shall incorporate the methods, means of access, and level 
of detail to be recorded for the various components of that bridge or class of bridges. 

3. The bridge inspection procedures shall ensure that the level of detail is appropriate to: 
a) the configuration of the bridge; b) conditions found during previous inspections; c) 
the nature of the railroad traffic moved over the bridge (including equipment weights, 
train frequency, train length, and levels of passenger and hazardous materials traffic), 
and d) vulnerability of the bridge to damage. 

4. The bridge inspection procedures shall be designed to detect, report, and protect 
deterioration and deficiencies before they present a hazard to safe train operations. 


Only eight state DOT rail bridge inspection manuals are available online. Those states are 
Alabama [3], Connecticut [48], Florida [11], Massachusetts [49], Michigan [50], Minnesota 


[22], Ohio [33], and Vermont [51]. Out of these, the state DOTs for Alabama, Florida, 
Minnesota, and Ohio use the same manuals for both highway bridge and rail bridge 
inspections. In comparison, Connecticut, Massachusetts, Michigan, and Vermont have 
developed their own railroad bridge inspection manuals. A brief summary of some specific 
aspects of their rail bridge inspection manuals is given here: 


Connecticut’s railroad bridge inspection manual [48] describes six different types of 
inspections. A description of these inspection types is presented below. 


1. Routine Inspections are regularly scheduled bridge safety inspections that are 
conducted every two years on all qualifying railroad structures as defined by the 
state’s Railroad Bridge Management Program. This inspection should include an 
inspection of the deck from the top, inspection of the bridge approaches, inspection of 
the underside of the deck, inspection of the bearings, inspection of the 
beams/superstructure, inspection of the abutments and wings, and inspection of the 
piers and waterway. A complete photographic record of the bridge shall be taken at 
each routine inspection. The required photographs are the same that are required for 
the in-depth inspection, and a list of the required photographs are listed in Section 
5.2.3 of the manual. 

2. Verification Inspections are regularly scheduled bridge safety inspections that are 
conducted every two years on all qualifying railroad structures defined by the 
Railroad Bridge Management Program. These verification inspections are performed 
on alternating calendar years from the routine inspections (Section 5.2.1) to satisfy 
the FRA requirement (49 CFR §237 Subpart E /52/) for annual inspection of railroad 
bridges in service. The general procedures for a verification inspection follow the 
guidelines for a routine inspection. 

3. In-Depth Inspections in compliance with current practice should be conducted on all 
qualifying structures every 10 years. An in-depth inspection consists of a hands-on 
examination of all exposed parts of a bridge to assess and record the physical 
condition of the bridge, to ascertain that the bridge is functioning as shown on the 
original plans, and to ensure that the bridge is adequate to safely carry the intended 
loads. 

4. Special Inspections are broken up into four subcategories, which include the 
following: 

e Interim inspections, to monitor a particular known or suspected deficiency. 

e Damage inspections are normally conducted immediately following any 
incident that may have an effect on the structural integrity of a bridge. 

e Flood survey—inspection, conducted after a major flood event. 

e Fracture critical and fatigue sensitive member inspections, to be performed 
together with the annual inspections. 

5. Underwater Inspections are conducted on any qualifying structure where the water 
depth around any of the substructure units is greater than 30 inches, and using hip 
boots and/or a raft is impossible or impractical because of poor underwater visibility, 
swift current, soft bottom conditions, accumulated debris, or low headroom. 
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6. Semi-Final Construction Inspections are conducted as bridge construction 
operations near completion and the contractor is still available to make corrections. 
On new structures, the entire bridge will receive an in-depth inspection. Semi-final 
construction inspections require close attention to detail and normally require a 
hands-on inspection. 


Massachusetts’ railroad bridge inspection manual [49] describes five types of inspections 
that are conducted by MassDOT on railroad bridges. The five types of inspections are 
periodic inspection, special inspection, fracture critical inspection, emergency inspection, and 
underwater inspection. These inspections follow the same procedure as the inspections found 
in the Connecticut manual. 


Michigan’s bridge inspection manual [50] does not outline any specific procedures for 
railroad bridges and only states that the NBIS provide the governing rules and regulations for 
the inspection of highway bridges located on all public roads that carry vehicular traffic 
throughout the entire United States. Although there is no governing state law or federal 
requirement to inspect structures which are not a part of the National Bridge Inventory 
(NBD), it is strongly recommended that each agency with designated responsibility of the 
traveled way perform systematic routine inspections to maintain the safety of the traveled 
way. 


Vermont’s railroad bridge inspection manual [51] describes six types of inspections that are 
conducted by the Vermont Agency of Transportation (VTrans) on rail bridges. The six types 
of inspections are annual inspection, detailed inspection, scour/underwater inspection, special 
inspection, seismic inspection, and cursory inspection. Four of them — annual inspection, 
detailed inspection, scour/underwater inspection, and special inspection — follow the same 
procedures as described in the Connecticut manual. The two inspections that do not follow 
the Connecticut manual are briefly described below: 

e Seismic Inspections are required when the railroad is notified of an earthquake 
registering 5.0f or higher on the Richter Scale. As soon as possible after notification, 
all bridges within a 100-mile radius of the epicenter shall be inspected, unless 
otherwise directed by the railroad bridge engineer (RBE). 

e Cursory Inspections represent quick examinations of a structure to identify visually 
conspicuous defects. 


3.2 Exploring Challenges with Integrating 
Data from UAS Inspections 


Building on the 2016 Phase I UAS project conducted for MassDOT [1], the UMass research 
team expanded the Phase I literature synthesis with a survey of state DOTs in order to review 
the current state of the UAS applications for bridge and rail inspections. First, state DOTs 
that have either integrated, or are planning to integrate UAS into their rail and bridge 
inspections were identified. Then, the research team contacted state DOT officials 
responsible for UAS integration to collect additional information regarding their experiences, 
lessons learned, and anticipated challenges. 
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A total of twelve state DOTs were identified which have or which are planning to integrate 
UAS into their bridge and rail inspections: Idaho, Illinois, Iowa, Kansas, Michigan, 
Minnesota, Missouri, Nebraska, New Hampshire, North Carolina, Oregon, and Vermont; as 
well as two federal agencies, FHWA and FRA. In addition, it was found that two private 
companies (Union Pacific and Norfolk Southern) are using UAS for railroad bridge 
inspections. Below is a summary of UAS inspection-related activities conducted by the state 
DOTs, private rail operators, and academia: 


Idaho Transportation Department (ITD), in collaboration with the Utah State University, 
researched the use of UAS in under-bridge inspections for detecting fatigue cracking. The 
objective of the research was to find the best commercial off-the-shelf visual spectrum 
camera for fatigue crack detection to use with UAS platforms that were available to the 
research team. The cameras were tested under different light conditions. Initially, the testing 
was conducted in the lab under a controlled environment. The UAS was also used to do an 
actual bridge inspection on the Fall River Bridge in Ashton, Idaho. The conclusion from the 
bridge inspection was that detecting fatigue cracking by using image processing is feasible, 
but it requires further study in order to determine the potential limitations of the technology. 
Further conclusions and insights from the experiment can be found in the project report [53]. 


Illinois DOT (IDOT) has already tested and is planning to test further UAS in many different 
applications. These include construction and project documentation, infrastructure and asset 
management, traffic flow monitoring, survey and mapping, pavement condition assessment, 
and bridge and structure inspections. More details of such applications using UAS can be 
found in the presentation created by IDOT [54]. 


Iowa DOT used UAS to monitor area flooding and railroad safety crossings, test beyond the 
line of sight railroad inspections, and to survey highway and railroad rights-of-way with 
LiDAR/IR sensors. Iowa DOT also tested the use of UAS for bridge inspection, crash 
investigation, and traffic operations. A brief overview of the lowa DOT UAS activities is 
available online [55]. 


Kansas DOT (KDOT) in a collaboration with the Kansas State University Transportation 
Center, conducted a literature review and studied the potential implementation of within the 
DOT. It was found that there are seven applications where UAS could be implemented to 
increase safety, efficiency, and cost savings. The seven applications include bridge 
inspection, cell and radio tower inspection, surveying, road-mapping, high-mast light towers 
inspection, stockpile measurements, and aerial photography. Identified challenges associated 
with the use of UAS for these purposes relate to how to deal with the large, potentially 
overwhelming, volume of collected data. There are also regulatory restrictions that prohibit 
UAS flights over people who are not associated with the flights, as well as the requirement to 
maintain visual contact with the UAS device. Additional details on KDOT’s UAS program 
can be found in the final report from the study with Kansas State [56]. 


Michigan DOT (MDOT) has conducted tests of UAS for bridge inspections since April 


2015. Images taken with UAS were used to detect deficiencies in bridge decking for potholes 
and wear, and involved the use of RGB cameras, as well as infrared and LiDAR sensors. 
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Michigan DOT also developed automated spall detection and automated delamination 
detection algorithms. Additionally, the research team implemented a Digital Elevation Model 
(DEM) Hillshade to detect potential defects in the surface of bridge decking. Further 
information on the MDOT UAS program can be found at the PowerPoint presentation 
created by Michigan Technical Research Institute [57] and the reports by MDOT [58,59]. 


Minnesota DOT (MnDOT) implemented a UAS program and conducted several studies on 
UAS applications for bridge inspections and other transportation needs. The results 
demonstrated that UAS can be used safely and effectively on a variety of bridges under 
different weather conditions. It was also concluded that while UAS cannot resolve all 
challenges associated with such inspections, UAS can be extremely helpful to inspect bridge 
elements that are difficult to access with traditional inspection methods. In most cases UAS 
expedite the inspection process as well as increase the safety of the personnel involved in the 
inspections. It was also found that UAS platforms equipped with thermal sensors can 
effectively detect concrete delamination. Detailed information can be found in the MnDOT 
report [60]. 


Missouri DOT (MoDOT) completed a feasibility study on UAS applications for state and 
local agencies. The study found that MoDOT fell behind many other state DOTs in the 
adoption of UAS technologies. The study recommended that a UAS program be initiated and 
suggested that the bridge inspections serve as a priority area for UAS implementation. Bridge 
inspections were selected as a priority because at the time (2018), Missouri was ranked sixth 
nationally in the number of bridges with over 24,000 in operation [61]. 


Nebraska DOT (NDOT) conducted a pilot study on drone applications in bridge safety 
inspections. The study included 11 bridges of different types and sizes, subdivided into the 
following six test groups: long urban bridges along a major highway in a city; bridges over 
rivers that were close to the bottom of the bridge; a bridge over water that is long and high 
above the river; culverts; arch bridges; and a fracture critical bridge. The intent of the study 
was to evaluate the use of drones for compliance with the NBIS guidelines for efficiency, 
quality, safety, and cost, as well as with current Federal Aviation Administration (FAA) 
regulations. The NDOT study concluded that with the exception of fracture critical bridge 
inspections, all other types of bridge inspections could incorporate UAS. Further information 
on the NDOT is available in an NDOT magazine article [62]. 


New Hampshire DOT (NHDOT) partnered with the University of Vermont to research the 
potential of using UAS in the transportation sector to promote safety, efficiency, and cost 
savings. Several studies have been developed to determine what transportation applications 
within the department are best suited for UAS integration. Potential applications identified in 
the studies include accident reconstruction, aeronautics, construction monitoring, traffic 
monitoring, rail and bridge inspections, as well as rock slope inspections. A detailed review 
of the case studies can be found in the NHDOT update on the UAS research, published in 
May 2018 [63]. 


North Carolina DOT (NCDOT) partnered with North Carolina State University to evaluate 
the potential benefits of UAS for transportation applications such as structural inspections, 
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small area surveys, and rockslide assessments, among others. The research found that UAS 
cannot replace manned aircraft large area surveys, but can be used for smaller surveying 
projects, as well as for infrastructure inspections projects such as bridge and rail inspections. 
A review of NCDOT transportation-related UAS research activities is presented in their 
report [64]. 


Oregon DOT (ODOT) conducted a statewide study on UAS applications for bridge 
inspections. Test flights have allowed ODOT to identify UAS platforms and sensors most 
suitable for bridge inspections. Their final report, Eyes in the Sky: Bridge Inspections with 
Unmanned Aerial Vehicles, released February 2018 [65] outlined the results of the study. A 
major conclusion of the study is that UAS can be a highly beneficial tool in the inspection of 
many bridges and towers. Study recommendations provide suggestions regarding UAS 
platform selection, sensor types, and settings depending on the performed task, environment, 
and other factors. The use of UAS was found to be most effective for initial and routine 
inspections, and less effective for more complex in-depth inspections that require touching, 
probing, or scraping a bridge. 


Vermont Agency of Transportation (VTrans) has tested UAS capabilities on highway and 
rail bridge inspections. Based on these tests, it was found that on-board UAS sensors can 
collect more detailed visual data of rail bridges compared to equipment traditionally used for 
such inspections. In addition, it has been established that UAS implementation can 
significantly reduce the amount of labor and costs associated with bridge inspections. On the 
other hand, it was noted that significant increases in the volume of data collected with the 
help of UAS may create additional challenges for data storage and processing. Further details 
on the project outcomes can be found in the VTrans’ final report [66]. 


West Virginia Department of Highway (WVDOH) - During the course of the study, two 
team members from the UMass — Amherst research team participated in a routine bridge 
inspection of the Patrick Street Bridge in Charleston, WV. Multiple UAS were flown by 
certified pilots under the guidance of the bridge inspectors to gather visual data. In addition, a 
UAS was flown along the bridge’s piers and near the truss and road surface with a thermal 
sensor to attempt to identify areas that need further inspection. The data collected with the 
help of UAS technology is currently under review. 


Union Pacific Railroad has developed a Perceptive Navigation Technology (PNT), which 
allows drones to perform railroad bridge inspections in areas with limited or no GPS 
coverage, such as under large metal bridges and culverts. The company believes that this 
technology is the first step in producing fully autonomous drones. Union Pacific is currently 
working closely with the FAA to ensure compliance with current regulations. The company 
is also working with software makers to provide a nationwide live stream of performed 
inspections. The technology currently allows streaming of 30 live feeds at a time. For more 
information on the Union Pacific UAS research and their innovative navigation and 
communication technology applications see the article entitled “How America’s Top 
Railroad Learns to Fly” [67]. 
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Norfolk Southern Railway has collaborated with its industry partner HAZON Solutions to 
introduce UAS into its daily operations. This partnership has allowed Norfolk Southern to 
perform over 64 complete railroad bridge inspections since 2016. The HAZON UAS are 
capable of flying within 15 feet of the bridge structure to capture high-resolution images of 
all of the bridge elements. In addition, HAZON proprietary technology allows UAS to fly 
under and inside the elements of railroad bridges. More information on UAS-related work 
conducted by Norfolk Southern and HAZON Solutions can be found in an article in Railway 
Age, a rail transportation journal [68]. 


In addition to information collected from online sources and reports provided by state DOT 
contacts, the UMass research team reviewed the final report for the NCHRP (National 
Cooperative Highway Research Program) Project 20-68A, Scan 17-01 [69]. This project was 
conducted as a follow-up to the AASHTO (American Association of State Highway and 
Transportation Officials) survey of state DOT UAS-related activities conducted in 2016. 


The report describes best practices of UAS implementation among state DOTs related to 
pilot training, hardware and software selection, data collection and processing, and other 
UAS-related activities. It was found that the most useful sensors for UAS when conducting 
bridge inspections are a high-resolution thermal camera paired with a high-magnification 
optical zoom camera. In addition, the scan identified the optimal processing workflow of data 
collected with the help of UAS. The processing includes data collection, storage, use, 
application development, and dissemination. Finally, the report recommends that state 
agencies create an outreach program to better educate communities on UAS applications, in 
order to minimize potential public privacy concerns. 


The NCHRP Scan included 12 state DOTs that either applied or researched UAS in 
transportation by 2016 when the first part of the NCHRP study was concluded. The scanned 
states include California, Connecticut, Florida, Georgia, Idaho, Indiana, Kentucky, Michigan, 
Minnesota, North Carolina, Oregon, and Vermont. A brief summary of the findings from the 
NCHRP scan is provided below. 


California DOT (Caltrans) conducted a study on the potential of using UAS for steep terrain 
evaluations. However, Caltrans has not implemented UAS into daily operations and does not 
allow contractors or employees to use UAS for any DOT projects. 


Connecticut DOT (CTDOT) announced its intent to conduct research and test the use of 
UAS for routine bridge inspections. 


Florida DOT (FDOT) researched the concept of using UAS in high-mast pole and bridge 
inspections and how this can reduce the cost and time, along with increasing quality and 
safety of the inspections. 


Georgia DOT (GDOT) studied the feasibility of using UAS in applications such as 
congestion monitoring, traffic signal inspection, vehicle speed sampling, bridge inspections, 
and monitoring wildlife and airport flight paths to determine the economic and operational 
benefits. 
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Idaho Transportation Department (ITD) researched the potential of UAS for bridge 
inspections. 


Indiana DOT (INDOT) tested the potential of using UAS to monitor construction progress. 


Kentucky Transportation Cabinet (KYTC) studied the use of UAS to improve safety, collect 
data, and reduce costs across multiple KYTC divisions. 


Michigan DOT (MDOT) evaluated and tested different UAS platforms, applications, and 
sensors for bridge inspections and other transportation applications. 


Minnesota DOT (MnDOT) researched, evaluated, and demonstrated the use of UAS in 
routine, interim, and special bridge inspections. 


North Carolina DOT (NCDOT) established best practices and recommended policies for 
agencies using UAS for search and rescue efforts and surveying after a flood. 


Oregon DOT (ODOT) studied the capabilities and limitations of using UAS to inspect 
different transportation facilities, including bridges and communication towers. 


Vermont Agency of Transportation (VTrans) studied the potential of using UAS for several 
applications, including geomorphic assessment, construction management and phasing, 
resource allocation during disaster response, and cost decision support. 


Finally, it is worth mentioning that a couple of UAS-related research activities that target 
infrastructure inspections were recently conducted in academia. Research groups from the 
Northwestern University in collaboration with Carnegie Mellon developed the Aerial Robotic 
Infrastructure Analyst (ARIA), a UAS platform equipped with visual cameras and a LiDAR 
to model and inspect infrastructure, including bridges. Rather than just observing, ARIA 
actively constructs a semantically rich 3D model of the structure that will enable new 
methods of analysis [70]. A similar project was conducted at the Florida Institute of 
Technology. The main objective of that research project was to investigate the applicability 
of mobile LiDAR and visual sensors to help detect concrete cracks and displacement of 
railroad bridge components. Results from this initial research effort indicate the potential 
practical value from using UAS and sensor technology for bridge inspection purposes. The 
overall consensus was that this technology, which is still need some time to mature, has the 
potential to significantly impact performance, effectiveness, and safety associated with 
bridge inspections. Details can be found in the Final Report for Rail Safety IDEA Project 26 
[71]. 


3.2.1 Summary of Experiences Associated with UAS Integration into Highway Bridge 
Inspection 


The results of the literature synthesis indicate that there is great potential for integrate UAS 
into current bridge inspection procedures. The synthesis found that 12 state DOTs either have 
implemented or studied the integration of UAS into their highway bridge inspection 
processes. It was also found that UAS integration into highway bridge inspections can 
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provide significant savings of time for both DOT inspection crew and road users as well as 
reduce costs and improve the safety of operations. 


A brief summary of the potential level of UAS integration into current highway bridge 
inspections procedures is presented in Table 3. 


Table 3. Summary of potential UAS integration into current highway bridge inspections 


procedures 
Procedure Potential Level of UAS Integration 
Initial (Inventory) Inspection Moderate application to assist with bridge inventory. 


Current UAS can be used as an asset management tool and 
to create point clouds and CADD files for future needs 


Routine (Periodic) Inspection Comprehensive implementation. UAS can be utilized to 
develop CADD files and to assist bridge inspectors by 
gathering imagery from difficult to reach areas and to 
identify areas that need further inspection 


Damage Inspection Moderate application to assist with damage assessment and 
inventory. UAS with advance sensors such as thermal can 
help inspectors understand the magnitude of the bridge’s 
damage. 


In-Depth Inspection Limited application to assist with the location of 
problematic spots. UAS can be used to gather imagery in 
different spectrums to assist the inspector. 

Fracture Critical Inspection Limited application to assist with the location of 
problematic spots. UAS can be used to gather imagery in 
different spectrums to assist the inspector. 


Underwater Inspection Not applicable for current UAS technology. However, some 
underwater remotely operated vehicles can be used to assist 
with the inspection. 


Special (Interim) Inspection Moderate application to assist with deficiency spot 
monitoring. UAS can be used to gather imagery in different 
spectrums to assist the inspector. 


Complex Bridge Inspection* Limited application to assist with the location of 
problematic spots 
*This type of inspection is not specified by BIRM. 


3.2.2 Summary of Experiences Associated with UAS Integration into Rail Inspections 


Currently, there are no actual UAS implementations or research activities conducted on 
railroad track inspections because of the strict regulatory requirements imposed by the FRA. 
While a test study could be conducted to evaluate the feasibility of using drones for rail track 
inspections, such inspections would most likely not meet FRA requirements for a railroad 
track inspection at this time. However, UAS have a great potential to inspect railroad bridges, 
assess damage after natural disasters and less severe impacts due to the adverse weather 
conditions to confirm that the tracks are clear and ready for freight and passenger rail 
operations. In addition, they can also be used to perform general surveying and mapping 
activities, and to serve as a data collection tool to help evaluate the amount of work required 
for construction and right-of-way maintenance. 
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3.2.3 Examples of Use of UAS for Rail and Bridge Inspections 
Examples are provided in Appendix A and Appendix B. 


3.3 Developing Practical Procedures and 
Protocols for MassDOT 


3.3.1 Literature Survey on Standard Operating Procedures in Other Agencies 


A survey was conducted of state DOTs as a continuation of the Phase I project to document 
their drone activities. The major objective of this survey was to understand where other state 
DOTs stand in terms of developing standard operating procedures (SOP) for their drone 
applications. 


Only four state DOTs responded with details about their development of standard operating 
procedures. To collect more information, the research team expanded the survey on the 
Internet and found standard operating procedures from two more agencies (Navy & United 
States Marine Corps (USMC) and Piper Mountain Aerial). The survey results are shown in 
Table 4 as follows. The first column identifies agencies surveyed and year of SOP 
documentation. The second column highlights major topics / headings included in each SOP 
documentation. 


Table 4: Details of SOPs from state DOTs and other agencies 


Agencies Highlights of SOPs 

California DOT Definitions, Acquisition, Authority, Restrictions, Planning, Incidents, 
2018 Providers 

Kentucky KYTC Operator requirements, Control equipment, Flight modes, Fail safe 

2015 procedures, Testing, Training, Battery, Maintenance, Flight OPs 

North Carolina DOT Pre-, During-, Post-flight operations, Emergency, Perimeter 

2017 management, Accidents, Crew communication, External communication 
South Carolina DOT Sense fly user manual: Airframe, Control, Communication, Processes 
2014 and procedures, Operations (flight phases and emergencies) 

Navy & USMC Maintenance, Flight operations (planning, pre-, during, post-flight), 
2018 Emergency, Reporting 

Piper Mountain Aerial Definition, Administration, Safety, Training, Operating procedures, Pre- 
2017 /Post-flight actions 


The survey results suggest that there is not a uniform approach to standard operating 
procedures for UAS use by state DOTs. Some agencies basically derived the procedures from 
the user manual provided by vendors, while others developed the procedures based on safety 
and organizational needs. Overall, it appears that the importance of training, planning, 
operation, management, and maintenance are generally recognized by these agencies. In 
particular, pre-flight planning, during-flight operation, post-flight actions, and emergency 
procedures serve as the central piece of standard operating procedures. Based on what was 
learned from other agencies, a practical UAS operating procedure was developed for 
MassDOT by integrating the agency’s specific needs and requirements. 
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3.3.2 Practical UAS Operating Procedure for MassDOT 
The practical UAS operating procedure for MassDOT is provided as follows: 


1. BUSINESS DECISION 

A procedure is called for to define the process of business decisions on the use of UAS. For 
example, who is eligible to initiate an application? How does one file an application? Which 
parties are involved in reviewing an application? What criteria are used to approve an 
application? 


2. AUTHORIZATION AND AIRWORTHINESS 

The FAA posted a rule in the Federal Register requiring small drone owners to display the 
FAA-issued registration number on an outside surface of the aircraft. Owners and operators 
may no longer place or write registration numbers in an interior compartment. The rule is 
effective on February 25. The markings must be in place for any flight after that date. 


On October 5, 2018, the President signed the FAA Reauthorization Act of 2018. FAA is 
evaluating the impacts of changes in the law and how implementation will proceed. During 
the period toward full implementation of the Act, MassDOT, UMass and others will continue 
to follow all current policies and guidance with respect to: 

I. Recreational Fliers & Modeler Community-Based Organizations 

II. Certificated Remote Pilots including Commercial Operators 


3. OPERATOR QUALIFICATION 

A minimum crew of two is needed for operating the UAS. A crew consists of a Pilot-in- 
Command and Visual Observer. The FAA does not require the VO to hold a Remote Pilot 
Certificate. Prior to flight, the PIC and VO should establish the takeoff and landing 
procedures that will be followed. They also should establish any contingency procedures. 
For example, in the event of a fly-away, the PIC and VO should have established who will be 
responsible for each element of the flyaway emergency procedure. 


Typically, both persons are in the takeoff and landing area, but sometime there are 
circumstances where the VO being farther away is more practical. An example of this might 
be around a bridge inspection where the terrain below the bridge is limited and the landing 
area is small. In a case like this it could be more practical for the VO to be on the bridge 
deck where they can more easily judge the safe distances between the UAS and the bridge as 
well as monitor the airspace above and around the bridge. 


Pilot-In-Command. The Pilot-In-Command is responsible for flight planning and ensuring 
that the UAS will be operating within the boundaries of the COA and in weather conditions 
that permit safe operation. This responsibility is described in 14 CFR107.49 (see below). 
The Pilot-In-Command is also responsible for safe flight operation in both manual and 
automatic flight modes. 


Bridge Inspectors and UAS Crew. All those involved with the bridge inspection are 


considered to be Operations Personnel. All drone operations shall require a team of at least 
two people and ideally both will hold a remote pilot certificate. This will not be a 


19 


requirement to carry out a planned mission. Only one of these operators must hold a remote 
pilot certificate to be able to carry out a mission and must assume the role of Pilot in 
Command (PIC). In that role the PIC’s number one responsibility is to be focused and in 
control of the operation of the drone. The second operator will be considered the Second in 
Command (SIC) and/or Sensor Operator (SENSO). In this role the SIC/SENSO’s number 
one responsibility will be to operate the sensor being used, such as a high definition or 
infrared camera. 


As stated in 14 CFR 107.49, the Pilot-In-Commanzd is also responsible for the following: 


“§ 107.49 Preflight familiarization, inspection, and actions for aircraft operation. 

Prior to flight, the remote pilot in command must: 
(a) Assess the operating environment, considering risks to persons and 
property in the immediate vicinity both on the surface and in the air. This 
assessment must include: 

(1) Local weather conditions; 

(2) Local airspace and any flight restrictions; 

(3) The location of persons and property on the surface; and 

(4) Other ground hazards. 
(b) Ensure that all persons directly participating in the small unmanned 
aircraft operation are informed about the operating conditions, emergency 
procedures, contingency procedures, roles and responsibilities, and potential 
hazards; 
(c) Ensure that all control links between ground control station and the small 
unmanned aircraft are working properly; 
(d) If the small unmanned aircraft is powered, ensure that there is enough 
available power for the small unmanned aircraft system to operate for the 
intended operational time; and 
(e) Ensure that any object attached or carried by the small unmanned aircraft 
is secure and does not adversely affect the flight characteristics or 
controllability of the aircraft.” 


It should be noted that the FAA DOES NOT have a flight training requirement or even a 
practical requirement for Remote Pilot certification. However, with safety in mind, all 
agencies should develop their own in-house training requirements. Example of a training log 
is provided in Appendix D. 


Visual Observer. The Visual Observer must be competent to observe a flight for the purpose 
of assisting the Pilot-In-Command in avoiding air and ground obstacles; as well as providing 
ground situational awareness to the Pilot-In-Command. 


Visual Observer / Second-In-Command (SIC): 
A. Responsibilities: The Second-in-Command’s (SIC) primary responsibility is to assist 
the PIC in the safe and efficient operation of the aircraft while carrying out assigned 
duties. 
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B. Under the immediate supervision of the Director of Operations and Chief Pilot, the 
SIC will be assertive with the PIC to identify any situation that may affect the safe 
conduct of the Flight. 

C. The SIC will declare an emergency and assume all duties and responsibilities of 
command and conduct of the flight as dictated by immediate circumstances in the 
event the PIC becomes incapacitated. 

D. The SIC has responsibility to notify the PIC of any deviations from Certificate 
Holder’s policies, procedures or federal regulations. Both pilots have an obligation to 
contact the Chief Pilot if there is a disagreement on procedures. The SIC is charged 
with informing the PIC of any unsafe condition or improper handling which could 
place the aircraft in jeopardy. 


The responsibilities of the VO are described in 14 CFR 107.33 and are defined as follows: 


“§ 107.31 Visual line of sight aircraft operation. 
(a) With vision that is unaided by any device other than corrective lenses, the 
remote pilot in command, the visual observer (if one is used), and the person 
manipulating the flight control of the small unmanned aircraft system must be 
able to see the unmanned aircraft throughout the entire flight in order to: 
(1) Know the unmanned aircraft’s location; 
(2) Determine the unmanned aircraft’s attitude, altitude, and direction 
of flight; 
(3) Observe the airspace for other air traffic or hazards; and 
(4) Determine that the unmanned aircraft does not endanger the life or 
property of another. 
(b) Throughout the entire flight of the small unmanned aircraft, the ability 
described in paragraph (a) of this section must be exercised by either: 
(1) The remote pilot in command and the person manipulating the 
flight controls of the small unmanned aircraft system; or 
(2) A visual observer.” 


4. FLIGHT PLANNING 

In order to maintain the highest level of safety and to mitigate risk of a UAS operation, this 
report suggests establishing what is commonly known in aviation as Tier 1 and Tier 2 
control. 


Tier 1 control is the authority of UAS flight by the person that has been named as the UAS 
Director of Operations (DO). This person may also be the Chief Pilot and can even be the 
UAS PIC for a given operation. 


In Tier 1 control, the Director of Operations is responsible for accepting the mission request 
and determining the pilot(s) who will operating the drone as well as other ground crew and 
the required on-board equipment needed to carry out the mission. The DO is also the person 
who will manage the flight planning meetings and establish the needed levels of safety based 
on external factors such as type of airspace, airspace hazards, type of equipment flown, and 
pilot experience. 
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Tier 2 control is the authority given to the Pilot in Command who is directly responsible and 
is the final authority as to the operation of the small unmanned aircraft system. 


Pilot in Command (PIC): The Pilot in Command is responsible for the safe conduct of all 
ground and flight operations conducted during their flight. The PIC shall ensure that safe 
conduct by complying with the following list as a minimum: 

All applicable FARs. 

The specific qualifications and duties described in Chapter 3 of this manual. 

All other applicable procedures listed in this manual. 

Certificate of Waiver or Authorization Specifications and Requirements. 
Certificate Holder’s Administrative policies. 


MoO D> 


The complete outline of the PIC’s roles and responsibilities are described in 14 CFR107.19. , 
as follows: 


“8107.19 Remote pilot in command. 

a) A remote pilot in command must be designated before or during the flight 
of the small unmanned aircraft. 

b) The remote pilot in command is directly responsible for and is the final 
authority as to the operation of the small unmanned aircraft system. 

c) The remote pilot in command must ensure that the small unmanned 
aircraft will pose no undue hazard to other people, other aircraft, or other 
property in the event of a loss of control of the aircraft for any reason. 

d) The remote pilot in command must ensure that the small UAS operation 
complies with all applicable regulations of this chapter. 

e) The remote pilot in command must have the ability to direct the small 
unmanned aircraft to ensure compliance with the applicable provisions of 
this chapter.” 


In Tier 2 control, the pilot is fully authorized the make all go/no go decisions around the 
UAS flight. The PIC works directly with the DO to determine any specific safety issues that 
need to be address and to discuss the mission planning. In some unique cases where 
modifications to a UAS are needed, the PIC may also need to work with the individual or 
team making those UAS modifications so that the PIC can determine that the aircraft is safe 
for flight. The PIC is ultimately responsible for determining that the aircraft is safe for flight 
as described in 14 CFR 107.15. See below: 


“8107.15 Condition for safe operation. 

a) No person may operate a civil small unmanned aircraft system unless it is 
in a condition for safe operation. Prior to each flight, the remote pilot in 
command must check the small unmanned aircraft system to determine 
whether it is in a condition for safe operation. 

b) No person may continue flight of the small unmanned aircraft when he or 
she knows or has reason to know that the small unmanned aircraft system 
is no longer in a condition for safe operation.” 
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An example of this scenario might be the installation of a LIDAR scanner that is being 
mounted to a drone or other sensor that is not commonly used. By working with the 
engineers, the pilot can determine more effectively of the sensor(s) does not adversely affect 
the aircraft’s weight and balance or overall performance. 


A pre-mission planning meeting between the DO, PIC and others may include but not be 
limited to the following items: 

1. Mission Objective—This addresses the purpose of the UAS operation. When stating 
the mission objective with regards to a bridge or rail section inspection, the DO 
would describe the type of inspection needed. They will also describe the deliverable 
that the “client” has requested. This might include such details as: 

a) The resolution of the imagery that is needed for a bridge inspection, or 

b) The accuracy of the imagery needed for a survey mapping project, or 

c) The accuracy and density needed for a point cloud if a 3D model is being 
developed. 


It is important to describe this deliverable because it is the driving force behind 
choosing the aircraft that is going to be flown and even the pilot chosen for the 
mission. 


2. Game Plan/Mission Outline—In this part of the mission planning the team discusses 
just how the flight will be conducted. For example, if the flight is autonomous for a 
bridge inspection how will the gridlines be flown. Where will the waypoints be 
located around the bridge to ensure that the needed data is captured in a time/battery 
efficient manner in a way that also promotes safety. 


3. Safety Concerns—In some instances, the airspace may be within the lateral 
boundaries of a controlled airspace, which means a higher probability of operating in 
the proximity of manned aircraft. When addressing safety concerns here, the 
discussion should be focused on other factors such as invisible hazards like high- 
tension wires, tree branches. 

a) Inthe event of a UAS flight around bridges, other factors need to be considered: 

1. Ifthe bridge’s metal structure will adversely impact the drone’s stability and 
its ability to maintain safe flight lines. 

2. The potential loss of GPS signal when conducting operations under a bridge. 
GPS loss triggers a drone’s internal safety protocols to switch to non-GPS 
mode which means that the drone will not maintain position autonomously. 
Instead the PIC must be aware that the drone will drift with the wind. 

3. Ifthe flight is being conducted below a bridge deck, the PIC must be aware of 
the strong wind currents that can form around bridge piers (supporting 
structures). In some cases, piers can create a wind tunnel causing airflow 
increase. 

4. Water is also a safety concern because for most UAS that utilize sensors for 
collision avoidance cannot detect water due to its moving surface. A loss of 
GPS over the ground results in a drone being able to maintain position via the 
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downward facing vision sensor, a loss of GPS over water renders the vision 
sensor useless. 


b) In the event of a flight along rail corridors, other factors need to be considered as 
well: 


L: 


If the drone is operating within the vertical limits of its downward facing 
vision sensors, then the drone is also within a distance where the high 
concentration of metal rails can adversely impact the drone’s compass and 
ability to fly straight. A compass error can result in a fly-away event. 
Invisible hazards such a power lines can be more common along rail corridors 
because the area has already been clear cut of vegetation. Power lines are 
difficult to see through First Person View (FPV) screens. 

Because rail corridors are long and often narrow and in remote areas, it can 
also be difficult for the pilot to see other types of invisible hazards such as 
antennae guy wires and tree branches. Line of sight with the VAS must 
always be maintained as described in 14 CFR 107.31, as follows: 


“8107.31 Visual line of sight aircraft operation. 


(a) With vision that is unaided by any device other than corrective lenses, the 
remote pilot in command, the visual observer (if one is used), and the person 
manipulating the flight control of the small unmanned aircraft system must be 
able to see the unmanned aircraft throughout the entire flight in order to: 

(1) Know the unmanned aircraft's location; 

(2) Determine the unmanned aircraft's attitude, altitude, and direction of 
flight; 

(3) Observe the airspace for other air traffic or hazards; and 

(4) Determine that the unmanned aircraft does not endanger the life or 

property of another. 
(b) Throughout the entire flight of the small unmanned aircraft, the ability 
described in paragraph (a) of this section must be exercised by either: 

(1) The remote pilot in command and the person manipulating the 

flight controls of the small unmanned aircraft system; or 

(2) A visual observer.” 


Note that paragraph (a) states “unaided eye ... other than corrective lenses.” This 
means binoculars and telescopes cannot be used to extend line of sight distances. 


Finally, the best way to identify if the airspace has unknown hazards is to perform an 
on-site inspection before the mission. 


Administration—While this might appear straight forward, administration further 
addresses who is doing what in preparation for the flight. If the flight requires survey 
mapping of a bridge or rail section, then perhaps an engineer will design and program 
the flight mission for the pilot. If on-site maintenance is expected, the DO may also 
need to assign that person. All of this is begin done to reduce the workload of the 
pilot and help ensure safety. 
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5. Weather—The weather requirements for UAS operations are the same as those for 
manned flight which is visibility of the more than 3 statute miles and the aircraft 
cannot come with 500 below a cloud or 2,000 feet horizontal of a cloud. Although 
drones fly in areas much smaller than manned aircraft, this weather requirement, 
visibility in particular, may seem excessive, it is the law and it does promote safety. 
In most drone operations, the aircraft is never more than a few hundred feet away 
from the pilot. When discussing weather as a part of the pre-flight mission planning 
the DO needs to address with the PIC what the weather minimums will be. 


6. NOTAMS—Notices to Airman (NOTAMS) are essentially special airspace notices 
that may be concerning to pilots. The most common types are called Temporary 
Airspace Restrictions (TFRs). TFRs are created to protect the airspace around things 
such as: 

(a) Sporting Events & Airshows 

(b) Wildfires & Natural 

(c) Natural Disasters 

(d) Spacecraft launch 

(e) Presidential movement outside of Washington, D.C. 
During pre-mission planning and all the way up to the actual UAS launch. The 
airspace must be checked for NOTAMS. 


7. Crew Assignments—This is the responsibility of the DO but by making these 
decisions as a group are more productive, more efficient, and more transparent. Some 
of the crew assignments include but are not limited to: 

(a) Pilot in Command (PIC) 

(b) Sensor Operator (Senso) or Second-In-Command (SIC) 

(c) Visual Observer (VO) 

(d) Officer (SO) — Whose role is obvious but may not be needed on-site. 


A successful UAS operation should also have an assigned Director of Maintenance 
(DM) if multiple drones are operated within an agency or company. This helps 
ensure that the aircraft all being kept maintained properly. In most cases today in 
which an agency or company has less than 10 drones, the PIC may also serve this role 
since they are ultimately responsible for the safety of the aircraft anyway. 


8. Airspace Rules — This part of the pre-mission planning is focused on ensuring the 
class of airspace in which the UAS is being operated. Most UAS flights are 
conducted in Class G airspace, however, in the event of an operation in Class B, C, or 
D an airspace authorization or waiver is needed. Due to the fact that bridge 
inspections are state or federal functions, this report suggests that whenever possible, 
adjacent land-owners to a bridge should be notified of the use of a UAS in the area. 
Being transparent about the UAS operation is beneficial to everyone. 
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5. CHECKLISTS 

The use of checklists should be required for all flights because when operating an aircraft, 
manned or unmanned safety is critical and if critical item is not checked, the results can be 
catastrophic. Checklists should be required for all pre-flight, in-flight, and post-flight 
operations. MassDOT approved checklist or other approved checklists may be used. In single 
pilot UAS operations, the PIC is responsible for the challenge as well as the response 
functions of the checklist procedure. In dual Pilot operations, the “Pilot not flying” (PNF) is 
responsible for the challenge function and the “Pilot Flying” is responsible for the response 
function. 


During emergency operations, the PIC and SIC, if applicable, shall use the checklist for the 
appropriate emergency after positive control of the aircraft is assured. 
The PIC shall ensure that checklists are completed. 


6. FLIGHT PROCEDURES 

Each operator of a UAS should utilize a checklist that they feel addresses all of the items that 
need to be checked/inspection prior to launch to ensure safe operation. This report also 
suggests reviewing the checklist on a regular basis or as needed to make additions/deletions 
to the checklist. A review of the checklists by a person or people that are not a part of the 
agency or company is encouraged because a review by people outside of the chain of 
command allows for more openness in making suggestions. 


Checklists used for manned aircraft are typically outlines in a “Command/Response” format. 
This means that the checklist is usually set up into two major columns where the first column 
is the item needed to be checked and the second column is the response that should be said 
out loud as the appropriate response. Deviating from the appropriate response can cause 
confusion, so the proper response should be said. 


For Example: Command Response 
Fuel Check/Full 
Landing Gear Check/Down and Locked 


Similarly, checklists for emergencies also ensure that each needed item is checked. The 
disadvantage to emergencies with drones is that the “emergency event” occurs so quickly 
with little to no warning. Emergency procedures may incorporate a set of rules to deal with 
emergencies and accidents such as radio fail-safe, loss of control link, battery fail-safe, GPS 
fail-safe, fail-safe for other system component malfunctions, inclement weather, emergency 
landing, and emergency/accident reporting. However, there are still checklists for UAS 
emergencies. 


The following [Figures 1 and 2] are copies of the checklists that were developed at UMass 
Amherst. 
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Matrice 210/XT2--Normal Procedures 


Matrice 210/XT2--Normal Procedures 


Preflight UAV Setup Response Crew Check Response 
1. Registration Docs Valid 1. Pilot In Command Confirmed 
2. Aircraft Integrity Check 2. Visual Observer Confirmed 
3. SD Card Installed 3. IMSAFE Checklist Complete 
4. Payload Installed 
5. Battery Installed Takeoff Response 
6. Propellers Installed 1. Senso Brief Complete 
7. Propellers Secure Secure 2. PIC Confirm Senso Brief Complete 
Preflight Controller Response After Takeoff Response 
1. Tablet Mounted 1. PIC Control Aircraft Control Check 
2. Lightbridge Connected 2. Senso Recording Check 
3. Tablet Battery Level Full 
4. Airplane Mode On Cruise Flight Response 
5. Controller Battery Level Full 1. PIC Time Callout every 2 min. 
6. Controller On 2. Senso Battery % every 2 min. 
7. DJI Go App On 
8. SD Card Memory Status Check/Formatted Landing Response 
9. Antennae Positioned 1. Gimbal UP Position 
2. GPS Mode Set/Active 
Before Takeoff Response 3. Power Available/Required Checked 
1. Safe Area Clear 4. Safe Landing Area Confirmed 
2. Propellers Secure 
3. UAV Power On After Landing Response 
4. Data Capture Check 1. PIC Motors disarm Off/Disarm 
5. Payload Secure 2. PIC Safe Appraoch 
6. Failsafe Settings* Established 3. PIC UAV Power Off/Disarm 
*Return-to-Home Altitude (set); Remote Controller Signal 4. RC Controller Power Off 
ee ee ve 
6. Aircraft Secure for Travel Check 
7. Debrief Execute Partial Complete 
Environmental Check Response Securing End of Day Response 
1. Airspace Classification Classified 1. Tablet Off/Stow 
2. Airspace Hazards Identified 2. Controllers Off/Stow 
3 NOTAMS Identified 3. Propellers Inspect/Stow 
4. TRF's Identified 4. Payload Inspect/Stow 
5 Weather Within Limits 5. SD card Remove/Stow 
6. Non-Participants Identified/Notified 6. Gimbal Cover Mount Secure 
7. Aircraft Stow 
8. Aircraft Logs Update 
9. Debrief Execute Complete 


Figure 1: Normal Procedures Checklist 
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Matrice 210/XT2--Emergency Procedures 


Matrice 210/XT2--Emergency Procedures 


Ground Fire 


Response 


Uncommanded Fly-Away 


Response 


1. Crew & Bystanders 


Alert/Clear Area 


1. Line of Sight 


Maintain 


2. Motors Disarm 2. Throttle Full Power Climb 
3. Disconnect UAV Power If Able 3. Flight Mode Switch Cycle 
4. Fire Extinguisher P.A.S.S. 4. Contact Management A.S.A.P. 
5. Call 9-1-1 As Needed If Control ls Regained Go TO Emergency Landing 
6. Contact Management A.S.A.P. 
Loss of GPS Satellites Response 
Flight Abort Response 1. Flight Mode Switch ATTI 
1. ANY "Abort" Announce 2. Abort Flight Execute 
2. Camera Up 
3. UAV Land Lost Link Response 
1. Line of Sight Maintain 
Flight Abort Response 2. Flight Mode Switch Cycle 
1. RPIC Announce Emergency 3. Controller Power Verify On 
2. Camera Up If Controller is Off, Power It On 
3. Land Immediately 4. Antenna Position Check 
4. UAV Power As Needed 5. Return to Home Activate 
5. RC Controller Power* As Needed If Situation Persists, Go to Uncommanded Fly Away 


*Take Screen Shots if Possible for Records 


Medical Emergency Response 
Unplanned Auto Land Response 1. Safety Brief Reference 
1. Throttle Full Power Climb 2. Call Call 9-1-1 
2. Flight Mode Switch Cycle/ATTI 3. Operator Will Need: 
3. RPIC Fas ees Location of Emergency 
4. Regain Comm. Signal Attempt Persons Problem/Incident 
If Comm. Signal Returns Land A.S.A.P. Age of Victim 
If Unable to Regain Comm. Signal ee dala Conscious Yes/No 
Breathing Yes/No 
Return-To-Home Response Battery Temperature Low Response 
1.RPIC cenaiaiaat - 1. Aircraft Land A.S.A.P. 
2. Aircraft Maintain VLOS 2. Battery Remove & Replace 
3. Flight Mode Switch ATTI 3. Battery Supply Ensure They Are Warm 
4. Aircraft Manual Control 
5. Controller eae Battery Overheat Response 
6. RTH Function Cancel if Active/Land 1. Electrical Load Reduce 
2. Aircraft Land A.S.A.P. 


Be Prepared for Electrical Fire 


Figure 2: Emergency Procedures Checklist 
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7. POST-FLIGHT PROCEDURES 
The checklists shown previously includes the procedures for ensuring that the aircraft is 
secure, tured off, and ready for transport. 


Upon completion of the flight(s) and as soon as practical, it is best to conduct a post-flight 
briefing in order to review how the mission was flown. Ideally the DO should be included 
since they are responsible for developing program policies and procedures. 


The Post-Flight Briefing might address the following questions: 
Was the mission successful? 

Were there any planned deviations? 

Were there any unknown hazards? 

Were there any safety violations? 

Did everyone understand their role? 

Did the aircraft perform as expected? 

What should be done next time? 


Se Ne 


Periodic Inspection of the UAS 

What will help reduce the number of incidents and accidents with a UAS is the regularly 
scheduled maintenance of each aircraft. Even aircraft that are not flown regularly should also 
be checked regularly. A brand new airplane that sits outside for a year is not safer than an 
older plane that is flown every day. 


In most cases drones do not come with a published maintenance schedule & maintenance 
checklist. The FAA also states that in the absence of a maintenance schedule and checklist, 
the operator should develop their own. 


The following [Figure 3] is a suggested checklist form for a drone maintenance program. 
Maintenance should be conducted as frequently as possible. 
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M210 Maintenance Checklist Response 


Record basic details 
Structural Inspection: 
Clean chassis of mud and dirt 
Inspect chassis for cracks 
Check for loose screws 
Check propellers for damage 
Check propellers are free-spinning 
Check motors for debris and obstructions 
Check state of wiring and solder joints 
Check unit camera is clean 
Check landing gear condition 
Inspect antennae 
Check control station for faulty components 
Battery Check: 
Inspect charge for visible damage 
Inspect battery packs for bulges or leakage 
Charge all batteries 
Software/Firmware: 
Update drone firmware 
Update control station software 
Finishing Up: 
Forward mainteance report 


Figure 3: Maintenance Checklist 


The procedures and protocols for safely operating UAS has developed over time. Earlier 
UAS inspections around bridges involved flying to specific locations (at the bridge 
inspector’s request) to capture imagery. This method promoted flight paths that were often 
non-linear and made it difficult for the remote pilot to judge distances even with the help of 
Visual Observers. 


Today, when flying a UAS manually around bridges, the emphasis is on “straight line” 
fights. These types of flights allow for: 


1. Minimal thumb movements on the flight controller switch reduces workload and 
improves safety. 

2. Ability to maintain consistent distances from the structure which results in consistent 
imagery resolutions. 

3. Improved ability to judge distances between the aircraft and the structure 

4. Ability for visual observer(s) to remain stationary which allows for more effective 
communication between the Remote Pilot and VO(s). 

5. Improved ability for the Remote Pilot and VO to determine the aircraft’s orientation 
in flight because the aircraft is not changing direction as often. 


The other option, which is still being developed at the time of this report, is to program the 
aircraft in advance to fly around the bridge autonomously. This method requires accurate 
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coordinates of the bridge and the supporting piers. The advantage of this method is that the 
aircraft can be programmed to fly closer to the structure, which could improve imagery. The 
disadvantage is that flight plan changes in the field are more difficult. The following [Figure 
4] is an example of flight paths during a manual operation. 


Figure 4: Manual flight path 


3.4 Testing of Developed Procedures and Protocols 


3.4.1 Phase One: Pre-Flight Day Prep 


During this phase the bridge location is identified. The mission objective is discussed and the 
“deliverable” is agreed upon. UAV Deliverables can include: Orthomosaic Inspections of the 
bridge piers and superstructure as well as the road surface. Developing an orthomosaic of the 
bridge allows future flights to be performed autonomously by incorporating the bridge’s 
CAD file to be imported into drone inspections software. Additionally, the deliverable might 
be photo or video only or live streaming. It may be visual, thermal, or another spectrum. 


Once the type of bridge inspection is determined and what areas of concern are identified, the 
Remote Pilot then assess the location’s proximity to any FAA facilities and determines the 
plan of action. In some cases, a special Certificate of Waiver needs to be obtained from the 
FAA in order to conduct the UAV mission. 


If the location does not interfere with an FAA facility, then the next step is to determine if 
there are any other known risks to the flight. Flight risks include power lines, cell towers, or 
other “invisible” hazards. If possible, the DO or PIC should contact adjacent land-owners to 
notify them of the upcoming flight. 
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Once these assessments have been made, the Remote Pilot can then develop the plan of 
action around the flight itself. This includes the staging area, launch point, landing point, 
route of flight, altitude, emergency procedures, crew members & their roles. 


3.4.2 Phase Two: One Day Prior 


The day prior to the bridge inspection, all aircraft used in the flight are inspected. The 
firmware checked and the IMU (Inertial Measurement Unit) is calibrated. If using a thermal 
sensor, the camera should be installed, turned and the PIC should go through all of the menus 
and inspect the quality of the imagery to ensure that the camera is working properly. 


3.4.3 Phase Three: Day of Flight Safety Analysis 


On the day of the bridge inspection, the staging area is set up, and the flight crew meet. This 
includes where crew members should be standing, a review of how to communicate, what to 
do in case there is an error with the aircraft and the mission is halted. At his time bridge 
inspectors can confirm what areas of the bridge are of particular concern. 


The mission is also rehearsed with a walk through of the area to determine any other flight 
hazards. 


3.4.4 Phase Four: Flight and Acquisition 
Prior to launch, the SD (memory) cards are formatted, a final pre-takeoff checklist is 
completed and the aircraft is launched to conduct a low-altitude system/flight control check. 


Upon completion, the drone is landed briefly, and then launched again to start the mission. 


When orthomosaic work is conducted the Remote Pilot verifies that the flight path is correct, 
and that imagery data is being acquired. 


3.4.5 Phase Five: Quality Control & QA 
Per UMass Air’s SOPs, aircraft must return to the landing point before the battery reaches the 
“30% remaining” indication. 


After each flight, the SD card is removed, and the imagery is copied into a local computer to 
determine, image quality, image correctness, and correct geo-tagging* of the imagery. 


3.4.6 Phase Six: Transfer of Data & Processing 


During this phase, the Remote Pilot meets with the in-house processor to discuss the flight, 
and then transfers the data for processing. 


In the event that higher levels are accuracy are required, geo-markers are placed at the 
location prior to flight. These markers are then captured during the flight from the air, and 
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when the orthomosaic is begin processed in-house, the geo-markers are used to help 
calibrate/correct the geo-tagged images from the drone. 


3.4.7 Safety 


3.4.7.1 Safety Statement 

UAS operators, whether flying as employees for a government agency, or as outside (vetted) 
UAS contractors need to comply with the safety standards established by the agency in 
charge of the inspection and the Federal Aviation Administration. Above all, a culture of 
safety should be promoted through a comprehensive safety management system. Each UAS 
related employee plays an important role in developing and maintaining a “safety above all” 
culture. UAS related personnel are encouraged to take an active part in the overall safety 
management structure by discussing, evaluating and reporting situations that may contribute 
to an unsafe working environment; along with diligence in following standardized policies 
and procedures. All UAS personnel are expected to comply with all federal, state and local 
safety regulations. 


3.4.7.2 Safety Standards 
A. Itis the intention and policy of the agency to establish and operate within the highest 
of safety standards. 


B. UAS personnel must operate within the scope of all Certificate Holder policies and 
the Code of Federal Regulations (CFR). 


C. This operation is governed by the applicable parts of 14 CFR Parts 61, 91, and 107, 
and the Certificate of Authorization or Waiver approved by the FAA. 


D. Safety is an individual responsibility and will be enforced by all supervisory 
personnel; safety will come first in all operations. 


E. All ground and flight equipment will be kept in top quality and safe condition. 


F. Safety will be promoted by thorough training of personnel, and by using good 
judgment in conducting day-to-day operations. 


G. It is the responsibility of each person to bring to the immediate attention of 
management, any practice or operating condition leading to an unsafe situation. 


H. It is the responsibility of the agency to develop a flight training program in order to 
ensure and track each pilot’s competency, and the help monitor any issues before they 
arise. 


3.4.7.3 Flight Safety 
Prior to each flight, the PIC and SIC/VO will each complete a Flight Risk Analysis Template 
(FRAT) ((see Appendix C) to determine the overall safety of each flight. The FRAT will be 
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submitted to the DO and kept on file for review by a Safety Review Committee if needed in 
order to help determine safety trends. Prior to each flight, pilots should also complete the 
“IMSAFE” checklist (see the following): 


3.4.7.4 Fitness for Flight 

Being prepared for you job each day helps to ensure overall safety. There is a self- 
assessment checklist to assist pilots in determining their own physical and mental health 
before a flight. The I'M SAFE Checklist is taught early in flight training and is used 
throughout a pilot's professional career to assess their overall readiness for flight when it 
comes to illness, medication, stress, alcohol, fatigue, and emotion. 


3.4.7.5 ILM.S.A.F.E. 

The “IM SAFE” card is a personal checklist that ensures the following statement is valid: I’m 
physically and mentally safe to fly, not being impaired by illness, medication, stress and 
fatigue, alcohol, or emotion (Table 5). 


Table 5: I.M.S.A.F.E. checklist factors 


Factor Potential Impact 


Illness Even a minor illness suffered in day-to-day living can seriously degrade 
performance of many piloting tasks vital to safe flight. The safest rule is not to 
fly while suffering from any illness. If this rule is considered too stringent for a 
particular illness, the pilot should contact an Aviation Medical Examiner for 
advice 


Medication Pilot performance can be seriously degraded by both prescribed and over-the 
counter medications, as well as by the medical conditions for which they are 
taken. The FARs prohibit pilots from performing crewmember duties while 
using any medication that affects the faculties in any way contrary to safety. 
Stress. Stress from everyday living can impair pilot performance, often in very 


subtle ways. 
Stress and Stress and fatigue can be an extremely hazardous combination. Fatigue and lack 
fatigue (lack of | of adequate sleep continue to be some of the most treacherous hazards to flight 
adequate rest) safety, as it may not be apparent to a pilot until serious errors are made 
Alcohol Extensive research has provided a number of facts about hazards of alcohol 


consumption and flying. As little as one ounce of liquor, one bottle of beer, or 
four ounces of wine can impair flying skills. 


Emotion The emotions of anger, depression, and anxiety may lead to taking risks that 
border on self-destruction. 


3.4.7.6 Interdisciplinary Safety Committee 

MassDOT puts great faith in the synergistic ability of different specialties working together 
to achieve and maintain a safe workplace. For this reason, this report suggests that a Safety 
Committee be comprised of expertise from each of MassDOT’s specialties involved in UAS 
Operations. The Safety Committee (if established) will provide quality and safety 
information, recommendations and concerns to the UAS Director of Operations through the 
Aviation Safety Director. 


34 


A. The ISC will include DO, a Safety Specialist, a Business Area Engineering Director 
(or designated director level alternate) and a highly experienced UAS operator. 


B. The ISC should convene on a regular basis to review UAS program safety, address 
specific incidents and accidents and determine corrective actions required. 


C. Any member of the ISC may request an out of cycle meeting to discuss critical safety 
issues as required. 
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4.0 Conclusions 


Central to the conduct of this research was a review of current procedures used in State DOT 
highway bridge and rail inspections and the experiences of these State DOTs in integrating 
UAS technologies into such inspections. Based on this review, the UMass research team 
concluded that UAS can serve as a useful tool for MassDOT in a majority of highway ridge 
and rail inspection procedures, with the exception of in-depth bridge inspections, because 
these types of inspections require hands-on testing. 


The major factors that affect the success of UAS integration into the bridge and rail 
inspections relate to selection of the proper types of UAS platforms and sensors. It is 
recommended that a rotorcraft UAS platform be used for the majority of bridge inspections 
and fixed wing platform for general surveys of extended areas, such as railroad right-of-way. 


The most useful sensors for bridge inspections include thermal sensors to detect areas of 
bridge deck delamination and high-zoom visual spectrum cameras to facilitate the close-up 
inspection of joints, bolts, and welds and to identify delineations like stress cracks in the steel 
structures, bridge decks and other elements. Also, LiDAR sensors are recommended for asset 
management to provide high definition measurements of transportation infrastructure; to 
conduct right-of-way surveys; to create 3D models; and to detect the presence of 
transportation infrastructure elements such as bridges, light poles, and signs. 


UAS can also be implemented to assist with rail bridge inspections, construction, and general 
maintenance and right-of-way inspections. However, because of the current FRA regulations, 
UAS cannot be used for the annual routine inspection of railroad tracks. 


Finally, it should be mentioned that the vast majority of bridge inspections will likely be 
conducted under conditions with limited or no GPS signal. Hence, an effort should be made 
to improve UAS capabilities to perform automated inspections in GPS denied environments. 
Technical measures that can help to alleviate this problem include Real-Time Kinematic 
(RTK) equipment to enhance UAS navigation in an environment with poor GPS reception; 
various collision avoidance sensors; and drone platforms capable of operating in direct 
contact with the surveyed structure. 


On the basis of the conducted literature synthesis, the research team successfully 
demonstrated some practical procedures and protocols for UAS integration into bridge and 
rail inspections. Those newly designed practical procedures and protocols have been field 
tested and demonstrated their effectiveness. However, these procedures and protocols are 
based on manual operations. The UAS pilots and the research team are in agreement that the 
preferred methodology for structures inspections (bridge and rail) is to utilize UAS software 
that would allow for autonomous flights. Similar to the reasoning behind the use of autopilot 
on manned aircraft, the use of pre-programmed UAS operations would improve safety. As 
autonomous software for UAS becomes more advanced, we should expect to see it integrated 
into these types of missions. 
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The research team also recommends developing a pilot program for further evaluation of 
UAS applications in highway and railroad bridge inspections and in railroad right-of-way 
maintenance inspections. 
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6.0 Appendices 


Appendix A: An Example of Benefits of Rail 
Inspection with UAS 


UAS has been used in Massachusetts for rail corridor inspections. In a portion of rail in 
Wilmington, MA, approximately 1 mile was flown. During this mission, the UAS was flown 
to gather orthophotos that were “stitched” together and then converted into a CADD file. 
The benefits were: 


| 


2 


A large orthophoto helped rail workers address the location of vegetative 
overgrowth that was at risk of getting too close to moving trains. 

The same large orthophoto was converted into a Normalized Difference Vegetation 
Index (NDVI) image to help rail workers identify the types of vegetation. The 
purpose of NDVI is to correlate color to plant species which will then allow rail 
workers to determine the frequency to cut back the vegetation. For example, a weed 
like poison ivy grows faster than an oak tree and both appear different in the NDVI 
spectrum. 

The orthophoto was converted into a CADD file with GIS overlay to help rail 
workers more easily determine the location of important rail related structures as 
well as adjacent properties. 


In Figure , the rail company created a cross section to show how the rail corridor should 
appear when looking down the corridor. Any vegetation in the corridor should be cut back. 
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Figure 5: Cross section of rail corridor and area for cutting back vegetation 
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In Figure 6, approximately 100 pictures were capture via UAS to help create this multi- 
stereo orthphotomosaic image or “ortho”: 


Lawrence's 


Figure 6: Multi-stereo orthphotomosaic image of rail corridor 


In Figure 7, the same ortho is converted to NDVI to correlate the color to the type of 
vegatation: 


Figure 7: Ortho image of rail corridor with vegetation differentiation 
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In Figure 8, the same section of rail with GIS overlay for workers to more easily use: 


Figure 8: Same section of rail with GIS overlay 
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Appendix B. Examples of Bridge Imagery 
Collected with the Help of UAS 


Modern UAS RGB sensors are typically equipped with a sensor with a resolution of about 16 
to 24 megapixels. Thermal imagery typically has lower resolution (640x512 pixels) compare 
to RGB but offer other advantages that can provide valuable information about the bridge 
structure conditions. Figure 9 is the image of a pier under the Patrick Street Bridge in 
Charleston, West Virginia. The image was captured with DJI Phantom 4 Pro 20 MP RGB 
camera from a considerable distance. The bridge inspector was able to identify areas of 
concern that need further details. 


Figure 9: Bridge pier image captured by a UAS 
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Figure 10 is the same pier shown under thermal imagery, which provides additional 
important details about the conditions of a pier. This image was captured at sunrise before 
the concrete structure had time to warm up which could yield a false reading. 


Note: Image captured at sunrise 


Figure 10: Same bridge pier shown under thermal imagery 
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Figures 11 through 16 in this section show other images captured through UAS. 


Note: Image taken in Massachusetts. While background is overexposed, area in question is properly seen. 


Figure 11: Underside image of a rail bridge deck to check for corrosion 


fic. 


Figure 12: Image of deck bearings of the same rail bridge 
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Note: This image is actually an ortho photo made of several dozen photos. 


Figure 13: Image of bridge pier, New Jersey 


aa 
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Note:Image was captured in just a few minutes, without slowing traffic. 


Figure 15: Image of bridge bearing, Massachusetts 
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Note: The purpose of the video was for asset management during a refurbishment. 


Figure 16: Screen shot from video, New Jersey 
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Appendix C: UAV Flight Risk Assessment 
Conducted Pre-Flight 


Prior to each flight, the PIC and SIC/VO will each complete a Flight Risk Analysis Template 
(FRAT). See sample template (Table 6) on next page. 
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Table 6: Flight Risk Analysis Template (FRAT) 


Before each flight, assess each of the following conditions and assign a numerical rating 
of 1 to 5 in the right-hand (Rating) column. Add up the entries in the Rating column to 
obtain an overall risk estimate and see where it falls in the Green/Yellow/Red Risk Chart. 
Add up the entries in the Rating column to obtain an overall risk estimate and see where it 
falls in the Green/Yellow/Red Risk Chart. 


taken during the flight. Consider delaying flight until conditions improve and risk is 
reduced. 


Pilot 
Observer 
Observer 
Points 1 2 3 4 5 Rating 
Location Wide Open Open Confined Congested & 
& Remote Space but Space & Urban* 
Suburbs Suburbs 
Airspace Class E & | Class D*** Class Class B*** Special 
G Cr Use*** 
Crewmembers Pilot & Pilot & 1 Pilot Only 
Observers | Observer 
Drone Primary Primary Primary Unfamiliar Unfamiliar, 
Experience Drone &> | Drone &< | Drone &< Drone Drone 
100 hours | 100 hours | 50 hours Testing* 
Sleep in last 24 >7 hrs 6-7 hrs 3-5 hrs 3-4 hrs <3 hrs* 
hrs 
Alcohol past 8 O drinks 1 drink 8 drank < 8 
hours hours ago hours ago** 
UAV Condition Newand_ | Reliable & Fair & Fair & Damaged & 
Tested Inspected | Inspected | Uninspected | Uninspected* 
Visibility > 3miles >2 miles >1 mile <1mile <1/2 mile* 
Ceiling > 1,000 ft. 500 - < 500* 
1,000 ft. 
Wind Calm 5-10 mph 10-15 15-20 mph >20 mph* 
mph 
Hours in last 90 >20 hrs 15-20 hrs. | 10-15 hrs < 10 hrs <5* 
days 
TOTAL 
Green: No unusual hazards. Use normal flight planning and established personal 10-20 
minimums and operating procedures. 
Yellow: Somewhat riskier than usual. Conduct flight planning with extra care. Review 21-33 
personal minimums and operating procedures to ensure that all standards are being met. | OF a5 in any 
Consider alternatives to reduce risk row 
Red: Conditions present much higher than normal risk. Conduct flight planning with extra | 33-45 
care and review all elements to identify those that could be modified to reduce risk. If or a5 in any 
available, consult with more experienced pilot or instructor for guidance before flight. 2 rows OF 
Develop contingency plans before flight to deal with high risk items. Decide beforehand on | alcohol < 8 
alternates and brief passengers and other crewmembers on special precautions to be hours ago 


** Alcohol consumed in less than eight hours results in a cancellation of flight 


*A score of five in any three categories results in a cancelation of flight 


*“* FAA approvals required 
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Appendix D: Flight Training Records 


Name: Joey Bagadonuts Task Number: 1 2 3 4 5 
: Mission Area and} Systems RC ah 
pone Pre-Launch Beaks and oe Controller oe ica 
Preparation Procedures Check 
4/16/2019] 1-6 : | P4/P4P Ss Ss Ss s iS iS s 
4/16/2019] _7,8 : P4/P4P iS iS Ss} s iS iS Ss 
4/16/2019] _ 9,10 : | P4/P4P s Ss s s iS iS S 
4/16/2019| 11 : P4/P4P iS S Ss Ss iS iS S 
4/16/2019] _ 12,13 : 5] P4/P4P iS iS Ss s iS Ss Ss 
4/16/2019] 14,15 : P4/P4P Ss S Ss S iS S S 
4/16/2019] Review ; 5] P4/P4P s Ss Ss s iS S s 
4/16/2019] Review E P4/P4P iS iS Ss s iS iS s 
4/16/2019] Review : | P4/P4P Ss iS Ss s iS Ss S 
4/16/2019] 16-21 : P4/P4P Ss Ss s s S Ss s 
4/16/2019] 22-26 0.5 5.5| P4/P4P iS iS Ss s iS Ss s 
4/16/2019] _ 27,28 0.5 6| P4/P4P iS S Ss iS iS iS S 
4/16/2019] _ 29,30 0.5 6.5| P4/P4P iS Ss Ss s iS Ss s 
4/16/2019| 31 0.5 7| P4/P4P iS iS Ss} s iS Ss s 
Figure 17: Flight Training Record for pre-launch (p. 1) 
6 7 8 9 10 11 12 13 14 15 16 17 18 19 
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Figure 18: Flight Training Record for pre-launch (p. 2) 
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Flight Training Record for emergency procedures 
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Executive Summary 


This study of Assessment of Roadway Pavement Condition with UAS was undertaken as part 
of the Massachusetts Department of Transportation (MassDOT) Research Program. This 
program is funded with Federal Highway Administration (FHWA) State Planning and 
Research (SPR) funds. Through this program, applied research is conducted on topics of 
importance to the Commonwealth of Massachusetts transportation agencies. 


The main objectives of this research are twofold: (1) performing a review of existing 
Unmanned Aerial System (UAS) technologies for pavement condition survey, and (2) 
conducting a pilot study to evaluate the applicability of using UAS for pavement condition 
analysis. The literature review has been focused on both sensors that can be integrated with 
UAS and advanced algorithms for analyzing pavement distress data. Since most pavement 
distress data is in the form of images, the review of algorithms are concentrated on pavement 
image processing methods developed recently. Other than the literature review, several field 
trips were made to the Fitchburg Municipal Airport (FMA) in Massachusetts to assess the 
feasibility of using UAS mounted sensors to collect image data for evaluating runway 
pavement conditions. The collected FMA data was used in conjunction with some online 
pavement images to evaluate the crack detection capabilities of a well-known Matlab tool 
and two deep learning methods. 


Chapter 1 briefly presents the objectives and background of this research. The literature 
review results of UAS technologies and data analysis algorithms are detailed in Chapters 2 
and 3, respectively. Based on the literature review, the use of UAS for pavement condition 
assessment is still in its infancy with little experience and information available. For 
integration with UAS, photogrammetry appears to be the most popular technique, while the 
use of multi- or hyperspectral imaging is getting increasing attention. The integration of such 
sensors with UAS for pavement condition assessment has not been thoroughly explored yet, 
although it does show potential. The literature review also shows that there has been growing 
interest in applying deep learning methods for detecting pavement distresses from images. 
Many deep neural networks models have been developed for this purpose and have achieved 
considerable success. However, most of the methods are to identify and highlight cracked 
regions with a rectangular box, instead of identifying cracks at the pixel level. There is still a 
lack of consistent and well-accepted procedures to convert crack detection results into 
Pavement Condition Index (PCI) values. Additionally, to develop deep neural networks 
based crack detection algorithms, it is important to have a large database of pavement images 
with cracks clearly labeled. Compiling such a large database is time consuming, and transfer 
learning appears to be a possible solution to address this issue. 


Increasing a drone’s flight altitude will decrease the pavement image quality but reduce 
costs. A qualitative comparison of the drone images captured at different altitudes at the 
FMA suggest that flying the drone 50 feet above the runway seems to provide the best 
balance between efficiency and accuracy. For the camera used in this study, the 50 feet 
altitude resulted in a 0.16 inches per pixel ground sample distance. Both visual and thermal 
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cameras were used for data collection at the FMA, and the results suggest that thermal 
images do not seem to add value compared to images from the regular camera. 


The collected pavement images were used to evaluate the crack detection performance of a 
MATLAB toolbox called CrackIT and two deep learning methods. Overall, the two deep 
learning methods outperformed the CrackIT toolbox. Among the two deep learning methods, 
the U-Net has the advantage of generating crack detection results at the pixel level compared 
to the SSD Mobilenet_V1 network. Applying deep learning methods requires a large 
annotated pavement image dataset. To address this issue, an online pavement image database 
was used in addition to the collected FMA data. The modeling results suggest that U-Net 
performs well on the FMA testing dataset when trained using the online pavement images in 
addition to a small sample of FMA images. The trained U-Net model was also applied to 
some highway pavement images collected using a van-mounted laser scanning system and 
generated satisfactory crack detection results. Overall, the introduced U-Net is a promising 
method for image-based pavement crack detection for both airport runways and highways, 
and demonstrated great robustness and generalization ability. 


Going forward, additional research is needed to understand how to determine PCI values 
from the pixel-level crack detection results, and compare the PCI values with the assessment 
results of qualified engineers. Also, a programmable procedure needs to be established to 
generate reliable and consistent PCI outcomes from UAS images. Finally, this research 
demonstrated the feasibility of collecting pavement images using cameras mounted on rotary 
wing drones and extracting useful crack information using deep learning methods. In the 
future, fixed wing drones can be considered to further expand the coverage and speed of 
UAS based pavement condition data collection. 
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1.0 Introduction 


This study of Task B: Assessment of Roadway Pavement Condition with UAS was 
undertaken as part of the Massachusetts Department of Transportation (MassDOT) Research 
Program. This program is funded with Federal Highway Administration (FHWA) State 
Planning and Research (SPR) funds. Through this program, applied research is conducted on 
topics of importance to the Commonwealth of Massachusetts transportation agencies. 


1.1 Problem Statement 


Paved roads deteriorate over time due to traffic loading and environment. To capture the 
extent and severity of the deterioration, state agencies and municipalities conduct pavement 
condition surveys of their roadway network. The data collected during these surveys is 
converted into indices that are used to guide decision-makers on how to best distribute 
available funds to keep the roads in service and safe. 


The data collected during these condition assessments consist of the distresses present on the 
roadway (cracking, raveling, weathering, etc.) and roadway profiles. In general, condition 
surveys are conducted annually or biennially. However, this survey process of evaluating the 
condition of pavements can be expensive, labor intensive, and time-consuming. Many 
traditional road evaluation methods utilize measurements taken in situ along with visual 
examinations and interpretations. 


The use of unmanned aerial systems (UAS) offers new potential for pavement managers to 
conduct condition assessments of roads and airport pavement. UAS have the potential to 
evaluate large areas in significantly less time compared to the existing manual methods. 
Findings from the 2016 Phase I research performed for MassDOT (“The State of the Practice 
of UAS Applications in Transportation” and “Current Counter-Drone Technology Solutions 
to Shield Airports and Approach and Departure Corridors”) indicate that UAS technology is 
actively being investigated by a growing number of state Departments of Transportation 
(DOTSs) to creatively improve safety, reduce traffic congestion, and save on costs. According 
to a March 2016 survey by the American Association of State Highway and Transportation 
Officials (AASHTO), 33 state DOTs have or are exploring, researching, or the use of UAS to 
inspect bridges and assist with clearing vehicle crashes, among other innovative applications. 


1.2 Objectives 


The main objective of this research task was to understand the current state and potential 
future state of practice regarding the use of UAS technology for pavement condition 
assessment. It attempted to address the following major questions: 


e How are other agencies using currently available UAS technology for pavement 
condition analysis? 

e What are the current UAS platform and instrumentation requirements that are needed 
to perform pavement condition analysis? 

e What are the future UAS platform and instrumentation requirements for fully- 
automated pavement condition analysis? 

e What data is collected by UAS and how is the data converted into a usable format? 

e How is the UAS data reduced, and how does this data compare to the data collected 
by the current state-of-the-art conventional and semi-automated methods? 

e What are the costs and cost savings involved with the use of UAS technology for 
pavement condition analysis applications? 

e What improvements in safety, time, quality of data, and cost can be expected when 
comparing traditional (non-UAS) methods, to semi-automated and fully-automated 
UAS pavement assessment methods? 

e If successful, what is the broader impact of using UAS technology for pavement 
condition analysis and subsequent pavement maintenance, repair, and re-construction 
decisions? 


1.3 Report Outline 


This research consists of a literature synthesis and a pilot study of using both traditional and 
deep learning methods for pavement distress analysis. The literature synthesis has been 
divided into two chapters for clarity. Chapter 2 addresses material related to UAS technology 
for pavement condition analysis, and Chapter 3 summarizes methods to detect distresses in 
pavement images. Chapter 4 presents the results of the pilot study and Chapter 5 concludes 
this research. 


2.0 Types of UAS Technology for Pavement 
Condition Analysis 


The two main types of data collection system used by UAS for pavement condition 
assessment are photogrammetry and light detection and ranging (LIDAR). LiDAR is an 
abbreviation for light detection and ranging. 


2.1 Photogrammetry 


Photogrammetry is the oldest and most common type of data collection system used by UAS 
for pavement condition assessment. This method is a less expensive option than LiDAR or 
spectral imaging. UAS photogrammetry consists of a series of “bird’s eye view” two- 
dimensional photos taken from a relatively low altitude and combined into one large 2D 
photo (1). A three-dimensional point cloud can also be generated by using a special method 
known as stereo-photogrammetry (1). Photogrammetry is different from aerial photography 
in that it is used to generate a composite map-like image which allows for distances between 
objects to be measured (2). New software is constantly being developed with improved 
algorithms to more accurately and consistently assess the type, severity, and quantity of 
pavement distresses. Ahmed and Haas (2009) describes the data analysis algorithms by 
saying they “rely internally on one or more threshold values during processing or may need a 
pre-processing stage” (1). What this essentially means is that despite the improvements in the 
algorithms of this software, the process still requires a lot of user input and refining to be 
reliable (1). 


2.2 Light Detection and Ranging (LiDAR) 


The Hartsfield-Jackson Atlanta International Airport has been a large supporter in the use of 
UAS for runway inspections (2). A presentation put together by the airport, Michael Baker 
International, and Pond & Company gives a brief introduction into the concepts of data 
fusion along multiple data collection systems including LiDAR technology (2). The Atlanta 
airport defines it as a “surveying method that measures the distance to a target by 
illuminating it with a laser,” offering a secondary method to measure various distances within 
a project to ensure agreement with photogrammetric data (2). Similar software is found on 
vehicle-mounted sensors that assess pavement condition; these are used by various public 
agencies and private businesses to determine road profiles from close range. A benefit that 
pavement condition assessment could gain from UAS LiDAR technology is that its “data can 
filter to the ground through the vegetation as long as light can be seen from under the tree 
canopy” (2). This quality of LiDAR data presents the opportunity to perform data fusion and 
cross-comparison to compensate for reduced accuracy in photogrammetry due to sub-optimal 
lighting (2). One downside of the LiDAR onboard equipment is that it “still weighs a lot, thus 


making it a challenge to keep the drone under 55 pounds” (2). Smaller and lighter LIDAR 
equipment are being designed to address this issue. 


2.3 Case Studies of UAS Technology for 
Pavement Condition Analysis 


2.3.1 Airsight—Germany 


The Airbus Operations Division of the Hamburg-Finkenwerder Aerodrome hired a company 
called AirSight to conduct an UAS inspection of a 130,000-square-foot section of concrete 
on the aerodrome’s apron (3). Their drone took aerial photographs and was operated on a 
GPS-based programmed flight path with a minimum distance of 150 meters to the runway 
and 10 meters to parking aircraft to ensure safety (3). The company’s software then took four 
days combining all of the photos into what they call a “single georeferenced orthophoto” 
with a resolution of 2 millimeters per pixel (3). Instead of using an autonomous interpretation 
of this dataset, several qualified engineers analyzed the single georeferenced orthophoto in 
AutoCAD and graded it based on a virtual-interpretation of ASTM (American Society for 
Testing and Materials) D5340 (3). The results were accurate, and the advantages include 
limited interruption to aerodrome operations, reduced human error or bias, and elimination of 
software-generated errors (3). This study proves the reliability of qualified human assessment 
for orthophotographs (3). However, this process can be time-consuming and more expensive 
than other options. No details were provided about the drone or equipment utilized. 


2.3.2 South Dakota 


The U.S. Department of Transportation sponsored a 2008 study done by South Dakota State 
University, whose findings demonstrated the ability of an UAS with a photogrammetric 
mapping center to assess the condition of an unpaved road (4). The airframe used was an 
Airstar International Mongoose airframe helicopter powered by a 26cc, single cylinder, 
Zenoah G260H engine.. The fight controller used was the combination of the Automatic 
Flight Control System hardware (AFCS) and custom Mission Control System software 
(MCS). The image sensor used was a UEye 2220c USB video camera with frame size 
768x576 with an on board OptiLogic RS-232 laser range finder. No details on the drone 
flight operations were provided. A GPS and geomagnetic sensor kept track of the drone’s 
position, velocity and altitude; which allowed for a pre-programmed flight path (4). A 
computer at ground-level was working in real-time with the UAS computer which “includes 
camera Calibration, integrated sensor orientation, digital 3D road surface model and 
orthoimage generation, automated feature extraction, and measurement for road condition 
assessment” (4). The main outcome of this reference was a proposed new strategy for 
assessing unpaved road conditions. 


2.3.3 Massachusetts Institute of Technology 


An MIT graduate student focused her 2016 master’s thesis on autonomous pavement distress 
detection through the use of drones. The experiment was conducted in small areas in 
Cambridge and Somerville, Massachusetts. A 3DR Solo Drone was used, flying at a speed of 
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1 meter per second at an altitude of 9 to 12 feet (5). The camera was a GoPro Hero4 (fisheye 
lens disabled) capturing 24 frames per second in 1080p resolution (5). Python API software 
was used to analyze the collected photos, while pre-programmed flight paths on Google 
Maps ran autonomously (5). The program converted images into grayscale, filtered them for 
smoothness to remove image noise (erosion, dilation, opening, and closing), segmented the 
data into a binary image to detect variations and inverted it for analysis (6). The next step 
involved autonomous estimation of contours, combined with elliptical regression (6). 
Distress geometry is calculated through analysis of texture, circularity, compactness, and 
size; then a distress classification is assigned (5). Software developers must work alongside 
pavement experts to tailor the digital image processing so that distress geometry and severity 
can be correlated to the Pavement Condition Index. This process is most valuable for 
identifying potholes and is not as useful for the identification of the many other pavement 
distresses because of the software’s generation of a high rate of false positives. (5). An 
onboard computer was mounted to the drone, but it was “unable to process the pothole 
classification algorithm fast enough for the tool to be realistically deployed in real time”; 
running about 10 times slower than would be required for real-time feedback (5). The 
hardware used in this study was sub-optimal, but the report gives an in-depth explanation of 
digital image processing. 


2.3.4 U.S. Air Force Institute of Technology 


The U.S. Air Force Institute of Technology prepared an extensive report in 2015, which 
discussed the viability of UAS paired with high-resolution cameras to assess pavement 
condition. The test area was a 2,000-foot-long airbase runway that displayed a wide range of 
various types and severities of pavement distresses (7). The size, type and resolution of 
camera used in this study were all taken into consideration, since higher megapixel cameras 
operate at slower maximum frame capture rates (7). This means that the UAS must fly slower 
to capture adequate data if a higher resolution camera is used (7). Furthermore, smaller 
capacity cameras must take more pictures to equalize the decrease in frame exposure time, 
which plays a role in determining optimal velocity (7). Another design constraint is battery 
life, which is directly related to flight time, velocity and quality and quantity of data collected 
(7). During the course of this study, a graph was created (shown in Figure 2.1) which shows 
the relationship between maximum flight velocity (while still attaining full coverage) and 
number of megapixels in a given camera (7). The software interprets the intensity of light 
values given off by pictures of the pavement and characterizes it within certain thresholds 
(7). Once these threshold values can be paired with corresponding distress types and severity, 
a reliable UAS Pavement Condition Index will be more attainable. The software has the 
capability of identifying pixels that correspond to a crack in the pavement and then assess 
whether there are any similar pixels within a certain range of pixels next to the already 
identified “crack-pixel” (7). This feature essentially maps the pixels of interest together, 
which can be used to identify alligator-cracking and longitudinal cracking (7). The software 
completes this task by “choosing the smallest distance edge and retaining it only if it 
connects a new node. Edges connecting already visited nodes are removed. Once all nodes 
are connected, the algorithm is completed” (7). Unfortunately, when a success rate for this 
method was graded by the F-Measure method, it displayed only 40 percent accuracy (7). The 
F-Measure method is a function of the recall and precision rates (both functions of the true 
positives and false positives and negatives captured by the algorithm) as compared to the 
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ground-truth image (7). Many different iterations and approaches were taken for analyzing 
this dataset to maximize accuracy, but spectral imaging may prove to be a useful tool for 
improving this type of assessment. 
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Figure 2.1; Maximum speed of image capture characterized by megapixels of camera 


2.3.5 Sidewalk Condition Assessment 


In a 2017 article submitted to the Institute of Physics Conference on Materials Science and 
Engineering, Ahmet Ersoz headed an experiment using UAS photogrammetry and automated 
software to assess the condition of a concrete pavement sidewalk. A DJI Inspire 1 
Quadcopter operating at different altitudes ranging from 0.5 meters to 3 meters with a speed 
of 1 to 3 meters per second was used with all photos stored on an onboard SD card (8). A 
digital image processing software known as an SVM (Support Vector Machine) used specific 
indexes like aspect ratio and eccentricity; but it still needed “training” from a human operator 
to hone in its threshold values (8). ASTM D5340 (Standard Test Method for Airport 
Pavement Condition Index—Surveys). Once the SVM model was properly trained, its 
assessment was compared to a “boots on the ground” field survey (8). The results were 
comparable; however, the author admits that the test area was an easy trial course since it did 
not exhibit a wide range of types and severity of distresses (8). They state that using only 
photogrammetric data collection leads to dramatically obscured data when there is light 
interference from shadows or different light angles (sensitive to location, time of day, and 
season) (8). Ersoz argues that “the main advantage of the system is that it offers a cost- 
effective solution compared to currently used systems such as a truck-mounted road 
monitoring system, as the UAS are getting cheaper and easily transportable” (8). He follows 
to say that the digital image processing algorithms still need significant improvement (8). 


2.4 UAS Platform and Instrumentation 
Requirements 


Hanson Engineering produced a brief overview on current UAS technology in 2016, which 
listed quadcopters and fixed wings as the two ideal types of drones (9). They cite two 
experiments that were conducted to assess the accuracy (compared to traditional surveying 
methods) of UAS photogrammetry for the purpose of creating a topographical map. The brief 
overview stated that UAS can be used for accurate mapping as well as for bridge inspections, 
roadway asset inventories, and airports pavement condition surveys. 


2.5 Data Collection and Reduction 


As noted in the previous case studies, data reduction is widely varied. Currently, automatic 
data reduction is still in the development stage. Most case studies utilized some manual 
interpretation of the results. 


2.6 Cost Savings 


No data was available on the cost savings of using UAS for pavement condition analysis. 


2.7 Benefits of UAS for Pavement Condition 
Analysis over Traditional Methods 


No comprehensive studies have been conducted to compare UAS over traditional pavement 
condition collection methods. 


2.8 Closing Summary of UAS for Pavement 
Condition Analysis 


The use of UAS for pavement condition surveys is in its infancy with very little experience 
and information available. Photogrammetry appears to be the front runner of UAS pavement 
condition assessment technology, but the use of multi- or hyperspectral imaging is garnering 
attention in this field of study as it presents more and more benefits when used in conjunction 
with photogrammetry. LiDAR usage in drones has not been thoroughly explored yet, 
although it does propose some potential advantages that may be seen at a later time. 
Understanding how to consistently determine a pavement condition index (PCI) is essential 
to this process. A better relationship must be established between qualified engineers who 
can reliably define and determine the PCI of a given section of pavement, and the 


programmers who are coding these algorithms; so that a more consistent system may be put 
into place. The “threshold values” and various assigned distresses need to be better integrated 
into a method that closely matches ASTM D5340. The “threshold values” and various 
assigned distresses need to be better integrated into a method for all types of pavements 
(highway, airport, etc.). 


3.0 Review of Deep Learning Applications in 
Pavement Distress Analysis 


Due to the various means of obtaining pavement images and the widely varied features 
associated with pavement distress, classification of pavement distress images presents a 
significant challenge for computer vision. Recent advances in computer vision and deep 
learning, however, have led to the development of deep-learning models for detecting 
distress in road pavement images. The models most recently used have been primarily 
centered on convolutional neural networks (CNN) (10). In this brief literature review, the 
current state of the art in CNN-based pavement distress detection is summarized. This review 
starts with discussing some of the key results noted in a recent review paper (10) on this topic 
and then examines a few other important papers that are not included in that review. 
Although these early studies are mostly for detecting the existence of pavement distresses 
and classifying them, the findings are still useful for developing a model that can detect 
pavement distresses at the pixel level and ultimately convert them into PCI measures. 


3.1 Summary of Gopalakrishnan’s Review 


In 2018, Kasthurira Gopalakrishnan published a review paper citing some of the most recent 
studies in the sub-field of CNN-based pavement distress detection and categorization (10). 
CNN is a special type of the well-known feedforward neural networks and is widely used in 
image analysis. Similar to feedforward neural networks, a CNN also consists of an input 
layer, multiple hidden layers, and an output layer. In feedforward neural networks, neuron in 
different layers are fully connected, meaning each neuron in a layer is connected to all 
neurons in the next layer. While for CNN, the hidden layers perform a sequence of 
convolution and pooling/subsampling operations on the input data, and are not fully 
connected. The results in some of the key works reviewed by Gopalakrishnan are 
summarized below. 


One paper reviewed by Gopalakrishnan (11) used deep learning to eliminate images of non- 
interest within a street-view image database. The NVIDIA Deep Learning GPU Training 
System (DIGITS) was used to perform this work. The author (10, 11) found that this deep- 
learning model was able to generate results that are 3 percent more accurate than the well- 
known CrackIT (12) tool. 


Zhang et al. (13) developed a CNN model to detect pavement cracks and tested it based on a 
dataset of 500 images of size 3264x2448 captured using a smartphone. These images were 
broken up by the authors into 1 million 99x99 RGB images. Of these 1 million images, 
640,000 were used for training the CNN, 160,000 were used for five-fold cross validation, 
and 200,000 were used for testing. This CNN utilized the ReLU activation function and was 
developed using Caffe. It made use of dropout in between layers in order to avoid overfitting. 


The stochastic gradient descent (SGD) method was used for model training with a batch size 
of 48. The CNN achieved the best performance after 20 epochs of training (10, 13). 


Pauly et al. (14) argued that a deeper CNN structure would present better results for 
pavement crack detection. They used the same 500 images as in Zhang et al. (13) and 
examined the effects of CNN depth on crack detection. The CNN was developed with four 
convolutional layers, four subsampling or max pooling layers, and two fully connected 
layers. The SGD optimization method was employed with a batch size of 48 for model 
training. The CNN was developed using Keras with a TensorFlow backend. Pauly et al. 
concluded that the deeper CNN architecture did increase pavement distress detection success, 
but did not present strong results when tested on images from new locations (10, 14). 


Eisenbech et al. (15) developed the open source GAPS (German Asphalt Pavement Distress) 
dataset, which consists of 1,969 grayscale images. They utilized 1,418 images for training, 51 
for validation, and 500 for testing. Initially, they utilized the same CNN that Zhang et al. (13) 
developed. Later, a deeper network structure was considered which consisted of eight 
convolutional layers, three subsampling or max pooling layers, and three fully connected 
layers. They tested several regularization techniques including dropout, batch normalization, 
and weight decay. The experiments took three months on a computer with two NVIDIA 
Titan X GPUs. The CNN was developed using Keras with a Theano backend. In general, the 
CNN achieved higher generalizability when employing the dropout or batch normalization 
techniques to avoid overfitting (10, 15). This result is not necessarily surprising, but it is 
important to be noted for future work in the field. 


Maeda et al. (16) developed a smartphone App called RoadDamageDetector to detect 
pavement distress. In their study, they distinguished between eight types of pavement 
distress. Their work utilized a smartphone mounted on a car dashboard. As the car was 
driven along the road at approximately 40 km/hr, the phone captured one image each second. 
These images, which were initially size 600x600, were downsized to 300x300 before being 
used for training the CNN. In order to develop the CNN, the Single Shot MultiBox Detector 
(SSD) with Inception V2 and SSD using MobileNet frameworks were utilized. The 
smartphone App was then developed on a Nexus 5X smartphone. The RoadDamageDetector 
App achieved a precision of greater than 75 percent (10, 16). The 9,053 images used in this 
study are also publicly available. 


Fan et al. (17) used images from two publicly available databases, the CFD database and the 
AigleRN database. To train the CNN, they broke up these images into overlapping 27x27 
patches surrounding each pixel. These patches served as the input to the CNN. They 
developed a CNN with four convolutional layers, two subsampling or max pooling layers, 
and three fully connected layers. All layers used the ReLU activation function except for the 
output layer which utilized the sigmoid activation function. The Adam optimizer was used 
with a batch size of 256 for model training, and a dropout rate of .5 was employed along with 
weight decay to avoid model overfitting. The CNN was trained on the AigleRN database for 
43 epochs and on the CFD database for 13 epochs. The CNN was developed with 
TensorFlow (10, 17). They conducted experiments on each database to confirm the CNN’s 
generalizability. First, they trained the network on the CFD database and tested it on the 
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AigleRN database. Then, they trained the CNN on the AigleRN database and tested it on the 
CFD database. Finally, they trained and tested the CNN on a hybrid database of images from 
both the CFD and AigleRN databases. The results showed good generalizability, especially 
when the network was trained on the hybrid database. They concluded that training on 
images with differing pavement conditions helps the CNN to achieve stronger 
generalizability (17). 


Rather than developing a CNN from scratch, Gopalakrishnan et al. (18) applied transfer 
learning to a pre-trained CNN. They utilized the VGG-16 CNN, which was pre-trained on the 
ImageNet database. They truncated the VGG-16 CNN, and only used the convolutional 
layers of the pre-trained CNN but not the fully connected layers. Then, a new fully connected 
layer was trained on the features recorded by the truncated VGG-16 CNN. They utilized 
1,056 images from the FHWA’s publicly available pavement performance database. The 
original images from this database, size 3072x2048, were downsized to 1000x500. The 
Adam optimizer was used with a batch size of 32. A dropout rate of .5 was also utilized. The 
CNN was trained for 50 epochs. They developed this transfer learning model using Keras. 
The best results were generated by a single layer classifier which was trained on the pre- 
trained VGG-16 CNN (10, 18). 


Gopalakrishnan et al. also showed the potential of utilizing pre-trained models with transfer 
learning for pavement distress detection based on UAS (drone) collected images (10,19). 
They conducted the analysis using a Keras implementation of the VGG-16 CNN. The size of 
the images in their study was 224x224. Dropout with a rate of .5 was used to avoid model 
overfitting, the ReLU activation function was used in the fully connected layer, and the 
Softmax activation function was used in the output layer. They trained the model for 50 
epochs with a batch size of 32. Their results showed that, by truncating the VGG-16 CNN 
before the fully connected layer and by adding a new fully connected layer, they achieved an 
accuracy of 89 percent in crack detection (19). The potential for using pre-trained models and 
transfer learning may be useful for this research. 


3.2 MatConvNet 


Cha et al. (20) reported using MatConvNet, an open source MATLAB toolbox for 
developing CNN, for the task of crack detection. They utilized 277 images of size 4928x3264 
for training and validation of their CNN. These 277 images were broken down into 40,000 
images of size 256x256 and were annotated as either containing cracks or not. They then 
designed a CNN using MatConvNet which they trained and validated on these smaller 
images. The CNN utilized the ReLU activation function and was trained using SGD with a 
batch size of 100. Both dropout and batch normalization were employed to avoid overfitting. 
For training the CNN, 32,000 256x256 images were used, and the other 8,000 were used for 
validation. The accuracies in the training and validation phases were 98.22 percent achieved 
at the 51st epoch of training, and 97.95 percent achieved at the 49th epoch of validation. 


Their CNN was also tested on 55 images of size 5888x3584 in order to determine if a sliding 
window technique would allow it to detect cracks in images larger than 256x256. This testing 
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showed similar results as in training and validating the CNN, with nearly 97 percent accuracy 
achieved in crack detection. Cha et al. also compared their CNN with the Canny and Sobel 
edge detectors and found that the CNN significantly outperforms these edge detectors for 
identifying cracks in pavement images. 


3.3 CNN for Crack Length Estimation 


Tong et al. (21) attempted to use a CNN not only for crack detection, but also for measuring 
crack length. In their study, the original 8,000 images were broken down into sub-images of 
size 200x200. Then, the gray-scale range of the images was cut, and gray-level histograms 
were computed. They found that the crack features in the images had an average gray-level in 
the range 50-110, while the normal pavement in the images had an average gray-level in the 
range 110-250. After the above pre-processing steps, Tong et al. developed a CNN which 
consisted of two convolutional layers, two subsampling or max pooling layers, two fully 
connected layers, and an output layer (21). 


Overall, the CNN was able to identify crack length in an interval of 1 cm (e.g., a 1-2 cm long 
crack, a 2—3 cm long crack) with an accuracy of 94.35 percent and a mean squared error of 
.2377. The model performed well for most 1-cm intervals, while the biggest issue was for 
classifying cracks with length in the ranges of 6-7 cm and 7-8 cm. This was likely due to a 
lack of images containing cracks of this length. 


The CNN was then tested on new images from four highways in central China. Forty images 
from each highway were acquired and pre-processed in the same way as the images used for 
developing the CNN. The results show that their CNN has promising generalizability (21). 


While the results presented by Tong et al. show the promise of using CNN for crack length 
identification, one step in their pre-processing method is unclear. That is, they were able to 
utilize gray-level histograms to distinguish between crack features and non-distressed 
pavement in their images. However, this result seems to contradict that of Chambon and 
Moliard (22), who found several years earlier that the histograms of pavement images which 
contain cracks have only one gray-level peak. Thus, Chambon and Moliard argued that the 
gray-level histogram of an image could not be used to distinguish a crack from non- 
distressed pavement (22). At this point, it is unclear why there is this discrepancy in the 
results of these two studies (21, 22). 


3.4 Closing Summary of Deep-Learning 
Applications in Pavement Distress Analysis 


This brief review shows that many deep-learning methods have been developed for detecting 
pavement cracks. While these methods utilize different deep-learning frameworks such as 
Caffe, TensorFlow, and Theano, it is important to note that the ReLU activation function 
seems to be the most commonly used. Also, stochastic gradient descent was employed by 
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most studies for model training, and it seems that dropout between layers is important for 
boosting CNN generalizability. The successful use of transfer leaning models by 
Gopalakrishnan et al. (18,19) may also be directly relevant to the current UAS and pavement 
distress detection work at UMass Lowell. Additionally, Tong et al. (21) were able to 
successfully use a CNN to perform crack length identification. While the study by Tong et al. 
presents important results for crack length identification using CNN, it is unclear how they 
were able to achieve such high success by using gray-level histograms to separate crack 
features from non-cracked pavement, and the generalizability of their model is uncertain. 


In summary, this review suggests that CNNs have been widely applied to the task of crack 
detection and present a promising way forward for pavement distress analysis. Also, the 
study by Tong et al. (21) shows the potential for CNNs in the task of pavement crack length 
measurement. Additionally, this review suggests that to develop CNN-based crack length 
detection algorithms, it is important to have a large database of pavement images with cracks 
clearly labeled. A key aspect of the work by Gopalakrishnan et al. (18, 19), transfer learning, 
appears to be an encouraging technique to address the limited labeled pavement distress 
sample size issue. Overall, the methods reviewed in this report are mostly for identifying and 
classifying cracked regions, not for directly measuring crack lengths. These methods, 
however, can be combined with traditional image processing techniques to isolate cracks 
from cracked regions and to estimate crack lengths. 


A very time-consuming step in developing CNN based pavement distress detection models is 
to label cracks in the training image dataset. The studies reviewed in this report used data 
from various sources, including smartphones (13, 14, 16), open source image databases (15, 
17, 18), and drones (19). It is possible to directly utilize the labeled images in these studies. 
However, this would restrict the new model to have the same input layer dimension as those 
models in the previous studies. Another issue with the existing datasets is that some images 
were taken by smartphones mounted on a dashboard. It is difficult to identify and label small 
cracks from such images manually. Therefore, the trained CNN models will not be able to 
identify them either. Lastly, these existing training datasets were labeled for identifying 
distressed regions (i.e., boxes were drawn around areas with cracks), while the ultimate goal 
of this research includes estimating crack length. Therefore, the existing datasets are useful, 
but still need to be further processed (i.e., label cracks only, not regions that have cracks) and 
expanded to cover additional crack features. 


13 


This page left blank intentionally. 


14 


4.0 Data and Analysis 


Based on the literature review, this research identifies and experiments with two methods for 
pavement crack detection. One is the MATLAB toolbox CrackIT (23) and the second one is 
a deep convolutional neural networks approach (24, 25). Both methods have been applied to 
the pavement image data collected from the Fitchburg Municipal Airport (FMA) in 
Massachusetts using drones and some additional datasets obtained from the internet. With the 
kind support of MassDOT Aeronautics Division, the team made two data collection trips to 
FMA on November 1, 2018 and March 1, 2019 respectively. Prior to these two data 
collection trips, the team received some pavement images collected by MassDOT 
Aeronautics Division using drones. For the first data collection trip, the team focused on 
evaluating the pavement images provided by MassDOT and comparing them with the 
pavement conditions in the field, and the team concluded that resolution of the received 
drone images is insufficient to provide useful pavement distress information (see Section 
4.1.1). In the March 2019 field trip, MassDOT Aeronautics Division helped the team collect 
a second set of pavement images using drones. The data and the analysis results are presented 
below. 


4.1 Data Sources 


4.1.1 Fitchburg Municipal Airport (FMA) Data 


Figure 4.1: Drone image from the initial data set 


The team obtained two sets of data from FMA. The first set of data was collected from an 
altitude of 120 ft and provided by the MassDOT Aeronautics Division. A sample image is 
presented in Figure 4.1. The team applied CrackIT and a deep learning method to this initial 
data set. However, the results were unsatisfactory. The team then visited the FMA and took 
some pictures of the pavement using a cell phone (see Figure 4.2). Clearly, the cell phone 
image covers a smaller area but is much clearer than the drone photo taken at 120 ft above 
the ground. The previous unsatisfactory crack detection results may likely be attributed to the 
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poor image resolution, not necessarily the methods adopted. Therefore, the team decided to 
collect additional pavement image data instead of trying to fine tune the existing CrackIT and 
deep learning models. 


Figure 4.2: Pavement image taken by a cell phone 


To investigate how height may affect the image quality, the MassDOT Aeronautics Division 
helped the team conduct another data collection at FMA. Table 4.1 below shows how the 
second set of data was collected. The elevator flight captured pavement images at a single 
location but at different altitudes using both visual and thermal cameras. The purpose was to 
find out how altitude may affect the image quality (e.g., inches on the ground per image 
pixel). With such information, the team was hoping to identify a threshold altitude, beyond 
which the collected data may become much less useful. Note that such a threshold may 
depend on the types of drone and camera being used. Therefore, the conclusion in this study 
may not be generalized to other cameras and drones. The coverage flight was designed to 
collect additional runway pavement images for crack detection model development and 
testing. 


Table 4.1: Flight plan for the second drone data collection 


Flight Description System Height (ft) Coverage 
Elevator Flight Using both DJI Matrice 210/XT2 10-70 (ata 5 ft interval), 80- Single Point 
Visual and Thermal Cameras Dual 150 (a 10 ft interval), and 200 
Coverage Flight Using a DJI Inspire 2/X4S 32, 40, 50, and 60 Half Runway 
Visual Camera DJI Inspire 2/X4S 70, 90, 120, and 200 Full Runway 


Figure 4.3 shows samples of pavement images collected at different altitudes through the 
elevator flight. It can be seen that once the altitude is greater than 120 ft, the pavement crack 
details are very difficult to see. This explains why the team’s previous modeling effort was 
unsuccessful. 


Choosing the drone flight altitude needs to consider both the image resolution and the 
efficiency. The FMA runway is about 150 ft wide. At the 10 ft altitude, it will take a drone 10 
runs to cover the entire width of the runway (each run covers a width of 15 ft). While at the 
50 ft altitude, the drone just need 2 runs (a width of 78 ft for each run). Figure 4.4 shows 
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Figure 4.3: Pavement image taken by the elevator flight 
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the relationships between flight altitude and field of view width and ground sample distance 
measured in inches per pixel (smaller ground sample distances mean better image quality) for 
the visual and thermal cameras used in this research. Note that both the visual and thermal 
cameras used in this study have the same ground sample distance performance. Again, 
depending on the cameras used, this conclusion may be different. However, the overall trend 
will be the same: higher altitudes will require less flights (i.e., better efficiency) but result in 
poorer image resolutions. To achieve a balance between image quality and efficiency, the 
team decided to use the images captured at 50 ft for this research. 
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Figure 4.4: Ground sample distance and field of view width vs. flight altitude 


During the elevator flight, the team collected images using both visual and thermal cameras 
and the results are compared in Figure 4.5 below. In the thermal image, pixels associated 
with pavement cracks do not appear to exhibit clearer characteristics compared to those in the 
visual camera image. For deep learning based crack detection models, cracks in the training 
images have to be marked at the pixel level. This process typically is done manually. 
Therefore, being able to identify cracks with naked eyes is essential. Given the quality of the 
collected thermal images, the team decided to only focus on the visual camera data. 


Thermal Camera 


Visual Camera 


Figure 4.5: Pavement image taken by the elevator flight 


4.1.2 Other Data Sources 


The team also obtained additional data from three other sources: 


e UMass Amherst Dataset (UAD): Prof. Charlie Schweik and his student Ryan Wicks 
helped the team collect pavement crack images using a MicaSense RedEdge M 
multispectral camera. The images were taken at the north-west corner of Parking Lot 
26 of UMass Amherst campus. A DJI Phantom 4 was used to carry the RedEdge M 
payload. The camera was mounted on a fixed platform with no gimbal such that the 
camera was angled forward approximately 10-15 degrees. 

e German Asphalt Pavement Distress (GAPs) Dataset (26): The GAPs dataset is 
captured using a mobile mapping system called S.T.I.E.R., which is manufactured 
and operated by a German engineering company LEHMANN + PARTNER GmbH. 
The S.T.I.E.R has been designed and certified by the German Federal Highway 
Research Institute (BASt) and complies with the German Road Monitoring and 
Assessment (RMA) standards. 

e Crack500 (27, 28): This dataset is collected by a group at the Temple University 
using a smartphone. It consists of 500 pavement pictures and each picture is of size 
3264 pixels x 2448 pixels. 


The GAPs and Crack500 datasets contain both the original pavement images and the 
annotated images as shown in Figure 4.6. The annotated images mark cracks at the pixel 
level and are essential for training/teaching neural networks models how to detect cracks. In 
Figure 4.6, black pixels in the second column represent cracks. Preparing the annotated 
images is very time-consuming. Therefore, it is beneficial to have datasets that include both 
the original and annotated images. 


The UMass Amherst dataset contains multispectral images taken at altitudes ranging from 20 
feet to 390 feet. Similar to the thermal images collected at the FMA, this Amherst dataset 
suggests that multispectral images do not seem to reveal more crack information than images 
captured by visual cameras. Considering the significant efforts needed to annotate the 
thermal images, the team decided not to include the Amherst data in the modeling process. 
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Figure 4.6: Sample original and annotated pavement images 


4.2 CrackIT for Pavement Crack Detection 


The team initially applied the MATLAB-based CrackIT toolbox to the initial FMA dataset. 
CrackIT provides many functions and routines to aid in the task of automated crack 
detection. A basic flow diagram describing the CrackIT method is shown in Figure 4.7. 
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Figure 4.7: Flow diagram of CrackIT detection method 


The toolbox only operates on grayscale images, so once an original image from the FMA 
dataset is read into MATLAB, it is converted to grayscale using the MATLAB function 
“rgb2gray.” Initially, the toolbox was applied “out of the box” on a sample image from the 
FMA dataset, and the detected results were poor. On the first attempt (Figure 4.8), CrackIT 


detected no cracks in the sample image. Only the black background region was returned with 
no detected cracks. 
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Figure 4.8: First attempt to apply CrackIT to FMA sample image 


After adjusting some parameters in the CrackIT functions (primarily those used for white line 
detection, pre-labeling of crack regions, and converting the image to binary), CrackIT 
returned a very noisy RGB map which contained little meaningful crack detection data 
shown in Figure 4.9. 


Figure 4.9: First attempt to fine tune CrackIT parameters 


There are some differences between the images in the CrackIT database and the images in 
the FMA dataset, such as the overall complexity of the images as well as the presence of 
many filled cracks in the FMA dataset as shown in Figure 4.10. Additionally, the images in 
the FMA dataset were captured using a drone, while the images in the CrackIT database were 
obtained using traditional road survey methods. Thus, it does not seem surprising that 
CrackIT “out of the box” did not perform well on the FMA data. 


To address this issue, sample images from the FMA dataset were resized and broken down 
into smaller chunks. The initial size of the images was 3648x4864 pixels. The images were 
resized to 3600x4800 pixels, and were then broken down into 192 non-overlapping sub- 
images of size 300x300 pixels as shown in Figure 4.11. 
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Figure 4.10: L: Sample image from the CrackIT database; R: Sample image from the FMA 
dataset 


Figure 4.12: Results of chunking image into 192 nonoverlapping chunks of 300x300 pixels and 
applying CrackIT on each chunk 


CrackIT was applied on each of these chunks individually, and the chunks were then brought 
back together to reconstruct a full RGB map of the cracks in the image. This divide-and- 
conquer approach improved upon the results of applying CrackIT “out of the box” on the full 
image from the FMA dataset, but the improvement is insignificant. In particular, there were 
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still many regions of cracks which were not detected, as well as substantial issues with 
noise/error in the RGB crack map as in Figure 4.12. 


To further simplify the image which was fed into the CrackIT algorithm, an eight-level 
thresholding was applied to the image before chunking. This eight-level thresholding, which 
is done utilizing the MATLAB function “multithresh,” maps each pixel in the image to one 
of eight possible pixel values, greatly simplifying the distribution of pixels in the image and 
in some sense de-noising the image as shown in Figure 4.13. 
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Figure 4.13: Comparison of image histogram before (L) and after (R) applying multilevel 
thresholding 


Once this thresholding was complete, the image was again broken into 192 chunks and each 
chunk was individually processed by the CrackIT algorithm. This eight-level thresholding 
step improved the results of applying CrackIT on the complex data from the FMA, leading to 
both increased and improved crack detection as well as a significant reduction in noise in the 
RGB crack map. A sample of the result is provided in Figure 4.14. 


Figure 4.14: Results of adding multilevel thresholding preprocessing step to CrackIT 
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A significant number of smoothing parameters have been tested to further improve the crack 
detection results. In each of the above trials, the smoothing parameters passed to the CrackIT 
anisotropic smoothing routine consisted of 4 iterations of smoothing using a conduction 
coefficient of 60 and a speed of .125. Further testing revealed that 2 iterations of smoothing 
with a conduction coefficient of 80, and a speed of .125 seemed to improve the crack 
detection results. Additionally, further testing at larger numbers of iterations with smaller 
conduction coefficients revealed that 16 iterations of smoothing with a conduction coefficient 
of 30 and a speed of .125 seemed to improve the results further, at least for some images, and 
at this point, these are considered to be the best parameters for smoothing. Figure 4.15 shows 
the results based on the best smoothing parameters. 


Figure 4.16: Error/false detection region 


24 


Some issues still remain. The first is the detection of the painted black lines adjacent to the 
white lines on the runway as crack regions. The second is the detection of false positive/error 
regions (see those blue, orange, and yellow regions marked in Figure 4.16) in the RGB crack 
map by the CrackIT algorithm. These false positive regions do not resemble any structures 
present in the original image, are generally square or rectangular in shape, and fairly densely 
filled. It is unclear why the algorithm detects these regions as pavement distresses. 


Because of the chunking method used in the processing of the FMA images, each of these 
false positive regions is generally contained only within one 300x300 chunk of the RGB 
crack detected image. For the step in CrackIT of converting the image to binary, CrackIT 
provides two possible options which correspond to two different MATLAB methods for 
converting to binary. If the method used for binary conversion in these regions of false 
positive is changed from “imextendedmin” to “im2bw” (a change which corresponds to 
passing a 1 to the CrackIT function “computeBinImg” instead of a 2), these regions of false 
positive are greatly reduced or eliminated. However, if this method for binary conversion is 
utilized on every chunk of the original image, then very few cracks are detected and the 
resultant RGB crack map returns a poorer representation of the cracks in the original images. 
Thus, at present, there is no consistent way to eliminate these regions of false detection/error. 


To further investigate these regions of false positive, a survey of several of these regions (one 
of them is highlighted in Figure 4.17 below) was conducted during a field trip to the FMA. In 
general, these regions of false positive do not show any signs of square-like distresses. Some 
of the regions show some signs of pavement wear, but this is inconsistent from region to 
region. Additionally, images of these regions were captured with cell phones. These cell 
phone images were processed using CrackIT, both using the parameters found to work well 
on the FMA dataset and using CrackIT “out of the box.” The results of a sample image are 
presented in Figure 4.18. CrackIT “out of the box” returned better results on these cell phone 
images, which is not surprising given that these images more closely resemble those in the 
CrackIT database. 


Figure 4.17: Example of false detection area inspected during field trip 


From Figure 4.18, it is clear that there are no square-like regions detected in these cell phone 
images. Upon inspection of the pavement at FMA and comparison with the drone images, it 
seems that the drone dataset does not present a completely accurate portrayal of the pavement 
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condition. For example, small, non-filled crack regions which are visible when walking on 
the pavement at the airport (and would be visible during a traditional pavement survey) are 
too small to be visible in the drone images from the initial data collection. A lower flyover 
height might help to improve the results of applying CrackIT to the drone images. 
Alternative smoothing methods, such as that of (29), have also been tested but have not yet 
improved the results. With further testing, alternative smoothing methods might also lead to 
improved results. 


Figure 4.18: a) Cell phone image of false detection area; b) CrackIT applied on cell phone 
image using best parameters from processing FMA dataset; c) results of applying CrackIT “out 
of the box” on cell phone image 


Some work on crack depth detection has also been conducted utilizing the Pix4D platform. 
Once a set of images from a drone flight are loaded into Pix4D, the software reconstructs a 
map of the terrain. From this reconstructed map, it is possible to calculate volume data for 
objects in the image. For example, utilizing a sample project provided with Pix4D, it is 
possible to calculate the volumes of houses and cars in the images. However, volume 
calculations of crack regions using Pix4D have not been successful, with the software 
generally computing a volume of 0 for any given small crack region. This difficulty is not 
surprising, given the large difference in the height of drone (e.g., 50 ft) flyover when 
compared with crack depth (e.g., 1 inch). Additionally, it is not surprising given the difficulty 
in visualizing open crack regions in the drone images. 
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4.3 Deep Neural Networks for Pavement 
Crack Detection 


The team also researched two deep neural networks approaches. The first one aims at 
detecting cracks using the transfer learning technique to avoid the time-consuming image 
annotation process. The second one is based on a promising deep learning approach called U- 
Net (24). The detailed modeling processes are described in the following two subsections. 


4.3.1 Crack Detection 


Developing deep learning models for object (e.g., crack) detection usually requires a huge set 
of training data. To address the lack of training data issue, transfer learning is often adopted, 
which is an important and useful means of adapting pre-trained deep learning models to new 
datasets, either when the new dataset is small, when prototyping time is important, or both. In 
(18) and (19) transfer learning was applied to crack image classification using the VGG16 
Convolutional Neural Network (CNN) as the base network. This network was initially 
trained on the ImageNet Database (31), and was used as the base network for training a crack 
image classifier. In this present work, transfer learning is applied to the task of crack object 
detection. The work reported in this subsection is based on the initial dataset collected at 
FMA. 


In order to accomplish this goal, many steps were followed. First, a platform and CNN was 
selected for transfer learning. In this work, the Tensorflow Object Detection API (32) was 
selected as the platform for training a new object detector on the initial FMA dataset. This 
API provided a variety of models which were pre-trained on various massive image 
databases such as COCO (33). In this work, the SSD Mobilenet_V1 network pre-trained on 
COCO was selected as the base network for training the new detector for the FMA data. This 
network architecture was also utilized for training a crack detector by Maeda et al. (30). 


In order to train an object detection network, images were labeled with ground truth 
bounding boxes which differentiate crack and non-crack regions. Additionally, the SSD 
Mobilenet network was trained using images of size 300x300 pixels. To prepare a dataset to 
develop the object detector, three sample images from the initial FMA dataset were resized 
from 3648x4864 pixels to 3600x4800 pixels, and were then broken into 192 chunks of size 
300x300 (just as was done above in the section on CrackIT and shown in Figure 4.11). Each 
of these 300x300 chunks was then labeled with ground truth bounding boxes using the 
Python tool LabelImg (34). An example is shown in Figure 4.19. Note that in Figure 4.19, 
the filled crack area is annotated by a rectangular bounding box. Therefore, utilizing this SSD 
Mobilenet network model, cracked areas were detected and marked also by a rectangular box 
instead of at the pixel level as in Figure 4.6. 
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Figure 4.19: Screenshot of utilizing LabelImg to draw ground truth bounding boxes 


Labellmg automatically generates xml files which contain the image annotation (bounding 
box) data. However, to utilize the Tensorflow Object Detection API, the image data must be 
in TFRecord format. Thus, these xml files of bounding box data were converted to TFRecord 
files, which could then be utilized for transfer learning. In total, 576 images were used, with 
490 (~85%) utilized for training and 86 (~15%) for testing. 


The first attempt to develop a model on this data was trained for 20,000 steps, utilized a 
constant learning rate of .0001, did not use data augmentation in the pre-processing, did not 
utilize dropout, and eventually attained a mAP (mean average precision) of ~.3176.' There 
seemed to be issues with the model overfitting the training data, with the testing loss 
ultimately at a high value of ~22. The results of this model are shown in Figure 4.20, in 
which the x-axes are for neural network training iteration, and the y-axes are for learning 
rate, MAP, and loss function value at each iteration, respectively. The learning rate, mAP, 
and loss function value do not have units. 
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Figure 4.20: From left to right: Learning rate, mAP, loss for the first attempt 


In order to deal with the issue of overfitting, data augmentation pre-processing steps were 
added to the training of the model in the second attempt. These included random flipping and 
cropping of the training data. After 100,000 training steps with an initial learning rate of 


' mAP is calculated at an IoU threshold of .5. 
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.000001, decaying to an ultimate value of 3.7735e-7, and utilizing dropout at a rate of .2, the 
network achieved a higher mAP of .4653 as in Figure 4.21. 
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Figure 4.21: From left to right: Learning rate, mAP, loss for second attempt 


To test the model further, a new random split of the training and testing data was conducted 
in the third attempt. A model was trained and tested on this new split for ~185,000 steps with 
a learning rate of .000001, which decayed every 5,000 steps to a final rate of 1.4989e-7. Data 
augmentation was utilized, as was a dropout rate of .2. Ultimately, amAP of .5867 was 
achieved on this split as in Figure 4.22. This was a further improvement over previous 
results. 


1.00066 0.600 
8,0008-7 + 8.00 
0.400 
6.000e-7 | 
7.00 
Rey 0.200 
2,0008-7 7 
6.00 
0.00 0.00 
RO CO ORE 18k 0.000 40,00k 80.00« 120.0k 160.0 et ee A ee 


Figure 4.22: From left to right: Learning rate, mAP, loss for third attempt 


Figure 4.23: Two example results of transfer learning: Right subimage of each pair is the 
ground truth, while left is bounding box predicted by neural network. 


One issue for the Tensorflow Object Detection API based crack detector is the detection of 
small, non-filled crack regions. The performance of deep learning methods depends heavily 
on the training data. There are significantly more filled crack regions present in the initial 
FMA dataset than non-filled regions. Therefore, the developed detector is unable to 
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consistently or accurately detect non-filled cracks. In order to continue to improve the results 
of the transfer learning object detection method, several potential steps may be taken 
including generating more ground-truth labeled (i.e., boxed) training data, continued 
hyperparameter fine-tuning, and possibly utilizing a different base network for transfer 
learning. Another issue is that the Tensorflow Object Detection API method only boxes the 
crack regions and does not tell how long or wide those cracks are. To address these issues, a 
U-Net deep learning model (24) is adopted and described in the next subsection. 


4.3.2 Crack Segmentation 


Given the promising performance of deep neural networks for image segmentation and object 
detection, this study applied a U-Net model to analyzing pavement images. U-Net is a CNN 
architecture initially developed for segmentation of neuronal structures in electron 
microscopic stacks. In its original application, it was utilized to learn a segmentation mask of 
neuronal boundaries (24) in Figure 4.24. 


Figure 4.24: Example of neuronal image and segmentation 


The task of segmenting neuronal boundaries is similar to the task of segmenting cracks and 
creating a map of the cracks in a pavement image. The Keras implementation of U-Net (25) 
was utilized in this work. The team started to experiment with the U-Net model using the 
initial set of FMA data. Since this U-Net network was designed for grayscale images of size 
512x512, all input images were first converted to grayscale. Some of these images were then 
resized to be broken up into non-overlapping blocks of size 512x512 if needed. 


Annotating pavement images for U-Net is at the pixel level, and it is even more time- 
consuming than labeling images for the Tensorflow Object Detection API in the previous 
subsection. Therefore, this study utilized the Crack500 dataset that consists of 2,244 already 
annotated images. In addition, this study selected some drone images obtained at the 50 ft 
altitude during the second FMA data collection trip. These images (5472 pixels x 3648 
pixels) were further divided into smaller pieces (512 pixels x 512 pixels), and 160 of them 
were randomly chosen. These new images were then annotated manually (as illustrated in 
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Figure 4.6) and used for model training and testing. Table 4.2 below shows how the input 
data was separated into training and testing datasets. 


Table 4.2: Data used in U-Net modeling 
Dataset Name | Total Images | Training Images | Testing Images 
FMA 160 70 90 
Crack500 2,244 1,896 348 


4.3.2.1 Measurement of Performance 


Crack segmentation is essentially a classification task at the pixel level. Furthermore, it is a 
difficult classification problem due to the highly imbalanced instances, i.e., much less crack 
pixels than non-crack pixels. To accurately measure the performance of the proposed model, 
the following F'1 score, Precision, and Recall metrics are presented first, and later an 
Intersection over Union (IoU) is introduced and used in this research. 


Precision * Recall 


iors ae Precision + Recall (1) 
— TP 
Precision = TP + FP (2) 
TP 
Recall = TP FN (3) 


Where TP denotes True Positive, FP denotes False Positive, and FN denotes False Negative. 
F1 considers both Precision and Recall, and is calculated as the harmonic average of 
Precision and Recall. F1 score value ranges between 0 and 1. Larger F1 scores mean better 
model performances. 


Many previous classification studies consider Precision as the main performance metric, 
which can be problematic. Precision may work if different outcome categories have 
approximately the same number of observations. However, pavement crack datasets are often 
characterized by a highly imbalanced class distribution. Based solely on the Precision 
performance metric, the best-performing model may favor non-crack pixels and generate 
poor results for crack pixels. Such a model does not provide much useful information for 
crack detection and may lead to biased if not erroneous conclusions. Therefore, F'1 (see Eq. 
(1)) is often used in recent studies. 


In this research, another intuitive and informative metric, IoU, is introduced and adopted. As 
shown in Figure 4.25 (35), JoU is defined as the area of intersection divided by the area of 
union. Intersection represents the area covered by both the ground truth and prediction, while 
union includes the area in either the ground truth or prediction. If the prediction matches the 
ground truth perfectly, then the corresponding JoU would be 100%. 
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Ground Truth Prediction 


Intersection 


Figure 4.25: Illustration of how IoU is defined 


4.3.2.2 Modelling 


Data augmentation was employed to increase the amount of training data for the U-Net 
model. For hyperparameters of tuning, The Adam optimizer was utilized with a learning rate 
of .0001. The number of training episodes was set to 1000, and binary cross-entropy was 
adopted as the loss function. ReLu activation function was used for all layers except for the 
last one, which utilized the sigmoid function. Additionally, the batch size was set to 5. 


Both the Crack500 and FMA datasets were used to train the U-Net model. To find out 
whether a U-Net model trained on the Crack500 dataset can perform well on the FMA 
dataset, two U-Net models were generated. The first U-Netcracksoo model was trained solely 
based on the Crack500 dataset using 1,896 images (See Table 4.2). The second U- 
Netcrack500&%FMA Model was trained using 1,896 Crack500 and 70 FMA images. 


The U-Netcracksoo model was initially evaluated based on the Crack500 data using the 
remaining 348 images set aside for testing, and its performance was very promising. This 
model was further tested on the FMA data and resulted in less accurate performance, which 
is not surprising and could be attributed to the differences between the two datasets. The 
Crack500 dataset was collected using a camera about 7 ft above the ground, while the FMA 
dataset was collected 50 ft above the ground. Given the limited size of the FMA dataset, 
training a U-Net model solely based on it is difficult. Therefore, the U-Netcracksooxrma model 
was developed and tested on the FMA data. The detailed model evaluation results are 
presented in the following section. 
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4.3.2.3 Results 


Table 4.3 presents the performance results of the two U-Net models and compares them with 
the results reported in a 2019 research paper that also used the Crack500 data. From the 
evaluation results based on the Crack500 data, U-Net clearly outperforms (JoU=0.6) the 
benchmark FPHBN model (28). 


However, the Crack500 data was collected using cell phones, while the FMA data was 
captured using a drone 50 ft above the runway. Due to the differences between the Crack500 
and FMA datasets, applying the U-Netcracksoo model directly to the FMA data did not 
generate satisfactory results (IoU=0.3). By combining only 70 annotated images from the 
FMA dataset with the Crack500 data for model training, the U-Netcracksooarma Model was 
able to produce significantly better performance (IoU=0.56) compared to the U-Netcracks00 
model on the FMA testing data. 


Table 4.3: Model performance comparison 


Evaluated Model Performance 
based on (IoU) 
Crack500 Benchmark FPHBN model 0.49 
from an IEEE 2019 paper (28) 
U-N etcrack500 0.6 
FMA U-Netcracks00 0.3 
U-Netcrack500&EMA 0.56 


The JoU results are useful for model comparision, since large values represent better 
performances. In addition, Figure 4.26 is included to visually illustrate the prediction 
performances of the U-Netcracksoo model on some randomly selected Crack500 testing data. 
Images from the first row are the original pavement data; images in the second row are the 
ground truth (i.e., true cracks); and images in the last row are the prediction results generated 
by the U-Netcracksoo model. Comparing the ground truth and prediction results in Figure 4.26 
suggests that the U-Netcracksoo model performs very well and is able to accurately capture all 
major cracks. 


Similarly, the prediction results of the U-Netcracksoo and U-Netcracksooxema models on the 
FMA testing data are further illustrated in Figure 4.27. A comparison of Figures 4.26 and 
4.27 clearly suggests that the Crack500 and FMA datasets are very different. Crack500 data 
has a higher resolution and differences between cracks and non-cracks can be easily 
distinguished. On the other hand, FMA data overall is darker and has a lower resolution. 
Some of the cracks are difficult to identify even manually with human intelligence. In Figure 
4.27, the third and fourth rows show the results of the U-Netcracksoo and U-Netcracks00%FMA 
models, respectively. Compared to the ground truth (i.e., the 2" row), the U-NetcracksoogEMA 
model clearly is able to identify more cracks than the U-Netcracksoo model. Given the limited 
time, this research only annotated 160 FMA images. If more annotated FMA images are used 
in U-Net training, its prediction performance is expected to be further improved. 
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Figure 4.26: Prediction results of U-NetCrack500 on Crack500 data 
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Figure 4.27: Prediction results of U-NetCrack500 and U-NetCrack500&FMA on FMA data 


Additionally, the developed U-Netcracksoo model was also applied to some pavement images 
generated by a laser pavement scanning system. Such a system is usually mounted on a 
vehicle (about 2 meters/6.6 ft above the ground) and used by many state departments of 
transportation to collect highway pavement condition data. The testing results are presented 
in Figure 4.28. The highway pavement images have not been annotated and the comparison 
is directly between the raw images and the predicted results. The highway pavement images 
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are taken about 2 meters/6.6 ft above the ground and are closer to the Crack500 data in terms 
of resolution than the FMA data. This probably explains why the U-Netcracksoo model’s 
performance on this dataset is very encouraging, although it is trained on a different set of 
images. This suggests that the U-Netcracksoo model has great generalization ability and can 
potentially be used directly by highway departments on datasets that are similar to the 
Crack500. 


~~ eoeqy 


Note: 1** Row: Original Images; 2"! Row:- Predictions 


Figure 4.28: Prediction results of U-NetCrack500 on highway pavement image data 


Figure 4.29: Application of U-Net to initial FMA data 


Additionally, the U-Net model has been applied to the first set of FMA images (see Section 
4.1.1) taken at 120 feet above the FMA runways. A subset of the images was used to train the 
U-Net model, which was evaluated on some separate testing images. A sample predicted 
image is provided in Figure 4.29. Again, the performance of U-Net appears to be very 
promising. 
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4.4 Closing Summary of Data and Analysis 


This pilot study collected runway pavement images using UAS from Fitchburg Municipal 
Airport (FMA). These images together with some online pavement image datasets were used 
to evaluate the crack detection performance of a MATLAB toolbox called CrackIT and two 
deep learning methods. Overall, the two deep learning methods outperformed the CrackIT 
toolbox. Both deep learning methods are able to generate satisfactory crack detection results. 
However, the SSD Mobilenet_V1 network only highlights the detected cracks with 
rectangular boxes. On the other hand, the U-Net model can detect cracks at the pixel level, 
which is an important feature needed for calculating PCI values. 


Deep learning methods typically require a huge set of annotated (labelled) training datasets 
for model development. This can sometimes be a major obstacle for the applications of deep 
learning models. This pilot study utilized some free pavement image data that has already 
been annotated together with a small set (70) of annotated pavement images captured at the 
FMA by a drone, and the trained U-Net model performed well on both the FMA testing data 
and some images collected using a laser pavement scanning system, suggesting that the 
develop model has great potential to be used for both airport runways and highway 
pavements alike. 
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5.0 Conclusions and Future Work 


This research consists of two major components: (1) a review of existing studies on 
pavement conditions analysis and related UAS applications; and (2) evaluation of promising 
methods for pavement condition data analysis. The literature review results suggest that the 
use of UAS for pavement condition assessment is still in its infancy with little experience and 
information available. For integration with UAS, photogrammetry appears to be the most 
popular technique, while the use of multi- or hyperspectral imaging is getting increasing 
attention. The integration of LiDAR with UAS for pavement condition assessment has not 
been thoroughly explored yet, although it does show potential. 


Besides the abovementioned sensing technologies, new algorithms for analyzing pavement 
condition data are also reviewed with a specific focus on deep learning. The results suggest 
that many deep neural networks models have been developed to detect cracks from images 
and achieved considerable success. However, most of the methods are designed to identify 
and highlight cracked regions with a rectangular box, instead of identifying cracks at the 
pixel level, which is an important step towards calculating pavement’s PCI value. To develop 
deep neural networks based crack detection algorithms, it is important to have a large 
database of pavement images with cracks clearly labeled. Transfer learning appears to be one 
effective way to get around this issue. 


The research also collects some runway pavement images using UAS from Fitchburg 
Municipal Airport (FMA). These images are used to evaluate the pavement crack detection 
performance of a MATLAB toolbox called CrackIT and two deep learning methods. Overall, 
the two deep learning methods outperform the CrackIT toolbox. Among the two deep 
learning methods, the U-Net model has the advantage of generating crack detection results at 
the pixel level. It also performs very well on the FMA data given a limited training dataset. 


Going forward, additional research is needed to understand how to determine PCI values 
from the pixel-level crack detection results, and compare the PCI values with the assessment 
results of qualified engineers. Also, a programmable procedure needs to be established to 
generate reliable and consistent PCI outcomes from UAS images. Recently, multi- or 
hyperspectral imaging has received much attention. The integration of such sensors with 
UAS for pavement condition assessment could be another interesting research direction. To 
facilitate highway applications, fixed wing drones can be considered to further expand the 
coverage and speed of UAS based pavement condition data collection, and their applicability 
needs to be evaluated. 
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Executive Summary 


This Evaluation of UAS Highway Speed-Sensing Applications was undertaken as part of the 
Massachusetts Department of Transportation (MassDOT) Research Program. This program is 
funded with Federal Highway Administration (FHWA) State Planning and Research (SPR) 
funds. Through this program, applied research is conducted on topics of importance to the 
Commonwealth of Massachusetts transportation agencies. 


Unmanned Aerial Systems (UAS) capability improvements and their ever-increasing use 
across industries provide an opportunity to revolutionize traffic data collection techniques. 
Previously, aerial studies of highway vehicle speeds were infeasible due to the high cost of 
helicopters, and because studies conducted at ground level could only capture speed data at 
specific locations along roadways. For example, MassDOT’s 2017 document “Procedures for 
Speed Zoning on State and Municipal Roadways” states that “it would be ideal to have speed 
checks at an infinite number of locations so that the 85th percentile speed could be computed 
at all points.” To address this need, the use of UAS to collect traffic speed data on roadways 
is being investigated. In addition to speed data collection, UAS have the potential to improve 
the efficiency and accuracy of other transportation data collection such as origin-destination 
studies. 


The objectives of this research are to: 

e Conduct a field study comparing the use of UAS to traditional speed data collection 
instruments on roadways in order to evaluate the feasibility of UAS as a traffic data 
collection tool; 

e Develop a methodology defining how aerial data can be utilized in the speed limit— 
setting process for surface transportation needs by using the data collected in the 
field; 

e Explore additional UAS traffic data collection uses for surface transportation needs 
beginning with an exploration of origin-destination studies. 


The drone used in this study was a DJI Phantom 3 Pro. This drone’s camera has a field of 
view (FOV) of 94 degrees and was flown at varying heights below the maximum allowable 
altitude of 400 feet. The altitude of each flight was chosen based on the minimum height to 
capture the full intersection or roadway segment. 


Using the data collected in the field and a literature review, a methodology was developed to 
understand the accuracy of the data and how it may be useful in the speed-limit setting 
process. Two studies were completed to understand the accuracy and cost of using UAS as 
compared to traditional methods: one for volume data collection and another for speed data 
collection. Using UAS and video processing, our method was found to have a count accuracy 
of 93% on average. Further, the speed data collection using our developed method had a 
relative error on average of 6.6%. At 50 mph, this error would be +/- 3 mph. For the speed- 
limit setting process, more detailed speed data may be required. However, it is noted that the 
speeds collected through our method were in the same range as the errors experienced 
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through the use of LiDAR and radar sensors, which are traditionally used today. These 
sensors have a range of error from -1 mph to +3 mph. 


Compared to traditional methods, our developed method has a similar upfront monetary cost 
of equipment. The large difference between the methods is the time-cost. One UAS flight is 
able to capture all of the vehicles passing through a location through the use of a video; 
however, LiDAR and radar sensors, when used manually, only collect one vehicle’s data at a 
time. Thus, UAS on medium to high volume roadways have the potential to be more time- 
cost effective than traditional methods. 


In order to achieve successful implementation of the developed methods, future work is 
needed. The algorithms and processes used to automate the analyses currently require a deep 
understanding of image processing and a strong proficiency in an engineering language, such 
as Matlab or Python. 


Future work should focus on training a diverse multitude of models so that the computer 
vision processing can be applied to any UAS video with minimal human intervention. These 
models would then need to be incorporated into a software program with a graphical user 
interface (GUI). Such a program and interface would enable any user to process their own 
UAS videos with minimal training and expertise. 
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1.0 Introduction 


This Evaluation of UAS Highway Speed-Sensing Applications was undertaken as part of the 
Massachusetts Department of Transportation (MassDOT) Research Program. This program is 
funded with Federal Highway Administration (FHWA) State Planning and Research (SPR) 
funds. Through this program, applied research is conducted on topics of importance to the 
Commonwealth of Massachusetts transportation agencies. 


Unmanned Aerial Systems (UAS) capability improvements and their ever-increasing use 
across industries provide an opportunity to revolutionize traffic data collection techniques. 
Previously, aerial studies of highway vehicle speeds were infeasible due to the high cost of 
helicopters and because studies conducted at ground level could only capture speed data at 
specific locations along roadways. For example, the Massachusetts Department of 
Transportation’s (MassDOT’s) 2017 document “Procedures for Speed Zoning on State and 
Municipal Roadways” states that “it would be ideal to have speed checks at an infinite 
number of locations so that the 85th percentile speed could be computed at all points”(1). To 
address this need, the use of UAS to collect traffic speed data on roadways is being 
investigated. In addition to speed data collection, UAS have the potential to improve the 
efficiency and accuracy of other transportation data collection such as origin-destination 
studies. 


The objectives of this research are to: 

e Conduct a field study comparing the use of UAS to traditional speed data collection 
instruments on roadways in order to evaluate the feasibility of UAS as a traffic data 
collection tool; 

e Develop a methodology defining how aerial data can be utilized in the speed limit— 
setting process for surface transportation needs by using the data collected in the 
field; 

e Explore additional UAS traffic data collection uses for surface transportation needs 
beginning with an exploration of origin-destination studies. 


Historically, speed limits were set based on the design speed of the roadway and then later 
adjusted based on the 85th percentile operating speeds of motorists traveling on that 
roadway. In recent years, this methodology has been called into question due to rising fatality 
rates among vulnerable road users (i.e., bicyclists and pedestrians). Expert systems have been 
developed to assist engineers and public officials during the speed limit-setting process 
better. While these systems do account for crash history and land use, operating speeds are 
still a key input. As such, there exists a need to improve upon speed collection procedures by 
developing a low-cost methodology to capture continuous vehicle speeds that would allow 
for precise locations of speed zone transitions to be known. 


Previous literature shows that small Unmanned Aircraft Systems (SUAS) are already being 
used for numerous civil engineering applications such as bridge inspection and crash 
reconstruction. This research seeks to expand sUAS applications to include speed data 
collection for the reasons mentioned above, while also including turning movement counts 
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(TMCs) and origin-destination (OD) studies. TMCs and OD studies are costly, yet important, 
early steps in transportation infrastructure studies. These studies provide information to the 
design engineers on the base traffic conditions of the roadway network. Such studies are 
typically performed using human technicians placed at the intersections being studied. sUAS 
have the potential to reduce the individual human hours, which can provide cost-savings for 
these studies. It is noted that all UAS in this section refer to sUAS, unless otherwise 
specified. This type of aircraft is less than 55 pounds and therefore only requires a Part 107 
license to operate commercially; heavier aircraft require more advanced operators and 
exemptions under the Federal Aviation Administration (FAA). 


1.1 Literature Review 


UAS have been utilized in the transportation industry in recent years to decrease cost and 
increase safety (2). This new lightweight, low-cost technology is portable and applicable for 
many different tasks, including bridge inspections, 3D mapping, and crash reconstruction (2). 
These devices are able to collect detailed information and capture aerial images with 
generally little amounts of effort and time. In recent years, UAS have begun to be 
appreciated for applications in traffic monitoring (2—7). Their ability to capture video above a 
roadway can be combined with object-tracking techniques to track vehicles and extract 
vehicle data such as speed, counts, and trajectory data (8-10). This data collection method 
can be useful for traffic engineering studies and can save time in the field, as UAS are able to 
collect large amounts of data in shorter amounts of time than traditional methods. In 
Massachusetts, the process for setting speed limits requires data collection through the study 
of many locations (1). MassDOT acknowledges that, ideally, observations are taken 
continuously throughout a proposed speed zone. However, in their most recent edition of 
“Procedures for Speed Zoning on State and Municipal Roadways” in 2017, MassDOT asserts 
that continuous data is not practical to collect (1). With UAS technology, continuous speed 
data collection becomes possible. 


Speed is a substantial contributor to crashes in the United States. From 2005 to 2014, 
speeding was a factor in over 112,000 fatalities, representing 31% of all traffic fatalities 
during that period (11). As speed limits promote roadway safety, they must be set reasonably 
and appropriately; they must reflect the roadway environment and driver expectations. If 
operating speed data is able to be collected continuously along a study area, providing more 
detailed data, then the operating speeds in that area would be better understood. With this 
higher level of understanding, it is expected that this data would result in speed limits that are 
more effective and promote increased safety. Overall, using UAS for speed data collection in 
the speed limit—setting process has the potential to improve safety and increase efficiency for 
the public agencies responsible for the process. 


1.1.1 Speed Limit Setting 


Traditionally, speed limits on newly constructed roadways are established from the design 
speed of the roadway segment. Generally, many speed limits have remained unchanged since 
they were set during original construction, but are no longer appropriate for the current 
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conditions. Speed limit modification studies are induced in different ways, including through 
town or city officials receiving complaints from the public or through an investigation of 
crash history. 


1.1.1.1 Speed Limit Selection Process 

State and local governments are responsible for the speed limit selection process for 
roadways in the United States (12). The National Cooperative Highway Research Program 
Report 500, which provides guidance on the Strategic Highway Safety Plan of the American 
Association of State Highway and Transportation Officials (AASHTO), states that a speed 
limit should depend on four factors: design speed, vehicle operating speed, safety experience, 
and enforcement experience (13). Design speed is based on a major portion of the roadway, 
not necessarily its most critical design feature, such as a sharp curve (13). As many design 
factors, such as adjacent land use and road type, for example, are based on anticipated use, a 
design speed does not always match the actual operating speed of a roadway (11). Vehicle 
operating speed is considered from a range of 85th percentile speeds taken from various spot- 
speed surveys of free-flowing vehicles at specific points on a roadway. The 85th percentile 
speed is widely recognized as the most utilized analytical method for selecting the posted 
speed limit as it includes many drivers’ speeds, or rather 85% of vehicles on a roadway are 
not exceeding that speed (11, 13). However, the National Transportation Safety Board 
concluded in its 2017 Safety Study that “the MUTCD (Manual on Uniform Traffic Control 
Devices) guidance for setting speed limits in speed zones is based on the 85th percentile 
speed, but there is not strong evidence that, within a given traffic flow, the 85th percentile 
speed equates to the speed with the lowest crash involvement rate on all road types” (11). 
Additionally, a 2016 Insurance Institute for Highway Safety report stated that the 85th 
percentile speed was not a stationary point, but rather a moving target that increases when 
speed limits are increased (14). 


Safety experience, or crash frequencies and outcomes, are also considered in the AASHTO 
guidance of the speed setting process (13). To consider factors other than operating speed, 
such as crash history, in a consistent manner, the Federal Highway Administration (FHWA) 
developed an expert web-based system, known as USLIMITS2. This tool is designed to help 
practitioners set “reasonable, safe, and consistent speed limits for specific segments of roads” 
(15). The input variables into the system include road function, crash history, pedestrian 
activity, and existing vehicle operating speeds. For engineers, the system can provide an 
objective second opinion (15). Enforcement experience is the final factor that is considered 
by AASHTO in the process of setting speed limits (13). 


Within the Commonwealth of Massachusetts, the process for establishing new speed limits 
depends upon roadway ownership (1). MassDOT procedures declare that in each case of 
exploring a new speed limit, an engineering study must be completed, which includes speed 
data collection based on free-flow traffic. The locations in which this speed data must be 
collected is dependent upon locality and uniformity of both physical geometric conditions 
and traffic conditions, but is typically spaced at intervals equal to or less than 0.25 miles (1). 
With a potential of long roadway sections of even just five miles or longer in need to be 
studied, the minimum number of study locations can be large. Currently, it is in general 
practice to collect speeds using a RADAR or LiDAR gun on the side of a roadway outside of 
plain view during weekday, off-peak hours under ideal weather conditions (1). These devices 
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can only collect speed at a singular point along a roadway. MassDOT acknowledges that 
ideally these observations would be taken continuously throughout a proposed speed zone. 
However, in their most recent edition of “Procedures for Speed Zoning on State and 
Municipal Roadways” in 2017, MassDOT asserts that continuous data is not practical to 
collect (1). At each study location, a minimum of 100 or more speed observations must be 
recorded in each direction; on low traffic volume roadways, observations may end after two 
hours if that value is not reached (1). Depending on the number of study locations, this can 
be a time-consuming and expensive process. LiDAR guns themselves cost $2,000 to $3,000 
(16). As a possible example, for a five mile stretch of roadway with 750 vehicles per hour, 
data would need to be collected at 20 locations. At each of those locations, the technician, or 
data collector, would have to manually use their LiDAR gun to record the speed of a vehicle 
and then write it down on their data sheet. In these specific roadway conditions and assuming 
that half of the vehicles travel at free-flow speed (375 vehicles per hour), 100 vehicles 
worthy of data collection would pass by the technician after just 16 minutes. However, not 
every consecutive vehicle can be recorded by the technician, as multiple vehicles may pass 
close to the same time. Given this, it is assumed that it would take the technician 
approximately 20 minutes to collect the data at a single location. Multiplying this by 20 
locations, it would take the technician approximately 6.7 hours of just data collection to 
obtain enough data for this type of study. It is noted that this time does not include travel 
time between the site locations and filling out summary sheets of each location. After field 
work, this data is now in paper form, so all 2,000 recorded speeds must be manually input 
into a computer and then checked by another technician. This may take approximately four 
labor hours in total. Thus, just for this fieldwork data collection process of the vehicle speeds, 
approximately 11 labor-hours are needed. 


For each study collection in the field, a “Sheet Distribution Worksheet” is required to be 
filled out with the following information: 95th percentile speed; 85th percentile speed; 50th 
percentile speed; mode; and pace (1). Further, a “Speed Control Summary Sheet” must be 
prepared at each study location, which requires all existing geometric conditions and 
constraints to be noted and mapped, including vertical curves, grade (if known), traffic 
volumes, side streets and major driveways, and adjacent land uses (1). Finally, among other 
factors in the speed setting process, such as crash history, the collected speeds are analyzed 
to create a safer speed limit for a given length of roadway (1). After a new speed limit is set, 
it is recommended in MassDOT procedures that a follow-up study be completed, requiring 
more time in the field and cost (1). 


1.1.1.2 Point Speed Capture Limitations in the Speed Setting Process 

Traditionally, speed data collection methods have utilized point speed capture, with 
continuous speed data considered impractical to collect (1). Point speed capture devices, such 
as RADAR, LiDAR, pneumatic tubes, and inductive loops, can each only collect speed data 
at a specific point along a roadway. As described above, the speed limit-setting process 
requires the existing operating speed along a study section of roadway to be fully analyzed at 
multiple points along the roadway. Utilizing point speed capture devices can be an expensive 
and time-consuming process over a stretch of roadway. Continuous speed data, if it is able to 
be collected along a roadway segment, would provide benefits such as inexpensive collection 
and a shorter turnaround time. Additionally, continuous data collection could provide new 
opportunities in the speed limit-setting process, such as determining specific locations where 
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the speed limit should change. Today, smartphone apps and GPS devices are able to capture 
this data; however, a shortcoming of this type of data collection is that it is not entirely 
limited to free-flow speeds, as there is a lack of information related to the time headway 
between vehicles (16, 17). 


As point speed capture data collection devices only allow for speed data to be collected at a 
single point along a roadway, only time-mean speed can be collected. According to the 
FHWA Travel Time Data Collection Handbook, time-mean speed is the “arithmetic average 
speed of all vehicles for a specified period of time” (Equation 1-1) (18). This differs from 
space-mean speed, which is defined as the “average speed of vehicles traveling a given 
segment of roadway during a specified period of time and is calculated using the average 
travel time and length for the roadway segment” (Equation 1-2) (18). In general, time-mean 
speed is associated with a point over time and space-mean speed is associated with a section 
of roadway. It is also stated that the direct collection of space-mean speed, rather than time- 
mean speed, is necessary to compute a theoretically correct speed (18, 19). 


Time-Mean Speed: Vrys5 = — = = (1-1) 
= d nxd 
Space-Mean Speed: Vsys = so = —— (1-2) 
p p SMS Ei Xt; 
where: d = distance traveled or length of roadway segment 
n = number of observations 
V; = speed of the ith vehicle 
tj = travel time of the ith vehicle 


1.1.2 Impacts of Speed Limits 


Speed limits are often a point of interest and controversy in a community. FHWA conveyed 
this aspect through their report “Methods and Practices for Setting Speed Limits: An 
Information Report” by stating, “Selecting an appropriate speed limit for a facility can be a 
polarizing issue for a community. Residents and vulnerable road users generally seek lower 
speeds to promote quality of life for the community and increased security for pedestrians 
and cyclists; motorists seek higher speeds that minimize travel time. Despite the controversy 
surrounding maximum speed limits, it is clear that the overall goal of setting the speed limit 
is almost always to increase safety within the context of retaining reasonable mobility” (12). 
In MassDOT’s own guide of procedures for speed zoning, this statement is referred to while 
reinforcing that speed limit setting is no easy task. This is why Massachusetts only 
establishes posted speed limits after an engineering study has been conducted (1). Thus, as 
many crashes are due to speeding as described in this section, speed limit-setting must be 
done with care to ultimately create the safest roadway environment. 


The National Highway Traffic Safety Administration considers a crash to be “speeding- 
related” if a driver was “charged with a speeding-related offense or if an officer indicated 
that racing, driving too fast for conditions, or exceeding the posted speed limit was a 
contributing factor in the crash” (20). In 2016, 27% of those killed in a crash involved at least 
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one speed driver. From 2015 to 2016, speeding-related fatalities increased by four percent 
from 9,723 to 10,111 (21). In the United States, speeding is a clear issue. However, speed 
limits cannot simply be changed to motivate drivers to operate at slower speeds. Speed limits 
must be set appropriately, as simply raising or lowering a posted speed limit without 
additional enforcement, educational programs, or other engineering measures, has little effect 
on the speed at which drivers will operate (22). Ultimately, people drive at a speed that they 
find reasonable and appropriate. If the engineers and agencies that set speed limits want 
drivers to respect speed limits, the speed limits must reflect the reality of the driving 
conditions. This cannot be done solely through enforcement, which could foster resentment 
instead of respect. Following proper speed limit-setting procedures and collecting accurate 
data can allow for appropriate speed limits to be set, creating a safer roadway environment. 


1.1.3 Traditional Speed Collection Techniques 


There are three categories of portable speed detector devices: intrusive, nonintrusive, and off- 
roadway (23). Appendix A—Traditional Speed Collection Techniques outlines examples of 
each of these types of technology, providing advantages and disadvantages. Each type of 
technology described in Appendix A are all point speed capture devices. 


1.1.4 Aerial Image Processing 


To collect more detailed information at a specific location, mounted video cameras can be 
placed to record the roadway. These devices are used in conjunction with video image 
processor systems to detect vehicles as well as specific data, such as speed. This technology 
has been understood and utilized for several years (24, 25). Processors analyze successive 
video frames to extract this data using algorithms and object tracking (25). Object tracking in 
video is often separated into three distinct areas: target representation, target 
detection/recognition, and target tracking (8). Each of these three components is described in 
further detail in this section. 


1.1.4.1 Target Representation 

To be able to accurately detect and track an object of interest, the system must understand 
what it is looking for under altering conditions. This can be accomplished through feature 
extraction, which plays a critical role in tracking (8, 26). Common visual features utilized in 
these algorithms include color, object boundaries, optical flow, and texture (26). Using these 
features, the goal of target representation is to create a model of the object of interest. This 
model includes the object’s appearance, size, and shape, along with some prominent features 
obtained from the feature extraction process in each image, which ignores the unuseful 
background in the image. This model can be created by extracting features on several 
thousand images of the object from a specific vantage point, or it can be selected either 
automatically online or by a user in the video sequence (8). 


1.1.4.2 Target Recognition 

Target recognition is the process of the target object being detected in a specific scene. To 
accurately detect an object, the model created in the target representation procedure must be 
used, and a search metric and a matching criteria to find this model by its features in a video 
frame must be defined (8). This search criterion separates the background from the object in 


a single video frame. This is done on every frame, with each frame considered one at a time. 
Some more high-level detectors utilize spatial information between several frames to detect 
an object, which reduces the number of misclassifications (8). 


1.1.4.3 Target Tracking 

Target tracking is the process of estimating the location of a particular target over a period of 
time (8). There exist multiple types of trackers, including point trackers, kernel trackers, and 
silhouette trackers (26). Kernel trackers, in particular, are commonly used to calculate the 
motion between frames. These trackers rely on an object’s appearance and shape. Point 
trackers, in comparison, track objects between neighboring frames described by defined 
points. For this type of tracker, a detection method must be applied to extract the points in 
each frame (8). 


1.1.4.4 Issues with Target Tracking 

The majority of nonstationary video tracking devices, such as Unmanned Aerial Vehicles 
(UAV), assume a flat world and that egomotion is translational (27). Fortunately, since UAV 
record video at relatively low altitudes (often less than 400 feet above the ground, due to 
FAA Part 107 regulations), the assumption of a flat world holds in almost all cases. The 
second assumption, that egomotion is translational, is true when minimal rotation occurs 
between frames at the feature level (27). Egomotion is defined as “any environmental 
displace of the observer” (28). In the case of a UAV in the sky, it is the assumption that the 
UAV will not rotate when recording video to use in the tracking process, so the movement of 
a feature can be classified as purely translational. Each of these assumptions simplify the 
processing necessary to stabilize UAV video, as well as show the importance of stabilizing 
the video while it is being captured (27). This can be minimized by flying UAV only in fair 
weather conditions. However, given the nature of UAV flying in the air, it is assumed that 
they will have some disturbance due to factors such as wind. To accurately take this into 
account, the vehicle motion that is tracked by the UAV is calculated as the sum of the real 
motion of the vehicle and that of the camera (4). 


Stabilization of the UAS and camera for accurate recording is an important issue to address 
when utilizing UAS for object tracking, as even a small wind force can lead to a large error 
on the ground. Stabilization of a camera on a UAS can be achieved through three steps: (1) 
using a gimbal, (2) completing a video analysis using a stabilization filter, and (3) tracking a 
stationary object through the entire video (3). Most UAV hold a gimbal for video 
stabilization. This allows for the rotation of the camera about a single axis, reducing 
movement while hovering. A stabilization filter eliminates camera shakiness while making 
panning, rotation, and zooming smoother. Finally, tracking a stationary object throughout the 
entire video allows for error due to camera and UAV movements to be taken into account, 
allowing the difference between the stationary object and object of interest to be calculated; 
thus, the movement of only the objects of interest can be separated. This is completed 
through the analysis of each video frame; at each frame, the difference between the 
coordinates of this specific stationary object can be applied to another object that is tracked 
(3). This final step is a way of considering that vehicle motion tracked by the UAV is the 
sum of the real motion of the vehicle and that of the camera (4). 


1.1.4.5 Commercial Video Processing for Traffic Data Collection using Mounted Cameras 
Many companies have commercialized automated vehicle tracking and traffic data 
processing across the globe. Miovision, for example, offers TrafficLink Detection to 
customers. This involves the installation of a single 360-degree camera at an intersection, and 
provides always-on turning movement counts, lane-by-lane volumes, and classifications for 
vehicle type (29). Other companies, such as Marr Traffic, Mike Henderson Consulting LLC, 
and L2 Data Collection Inc., conduct similar data collection through the use of mounted 
video cameras (30-32). 


1.1.4.6 UAS Video Aerial Image Processing 

One company that uses UAS-captured video aerial image processing for traffic data is 
DataFromSky, a company based in the Czech Republic (33). Their system only requires 
aerial video and a description of the scene to provide trajectories of every detected vehicle in 
the video. These vehicles are then labeled in the video by a unique ID, along with a record of 
the vehicle’s position, speed, and acceleration. DataFromSky is also able to analyze vehicle 
trajectories to calculate traffic flow characteristics that are defined by the 2010 Highway 
Capacity Manual (10). Additionally, they are able to provide gap acceptance, critical gaps, 
capacity, and capacity estimations, average speed, and vehicle counts. DataFromSky has 
partnered with several companies, including Traffic Analysis & Design, Inc. in the United 
States, who serve the states of Minnesota and Wisconsin. They have also cooperated with 
PTV Group to export results from DataFromSky and input it into PTV Vissim, including 
traffic counts, vehicle classification, turning movements, speeds for model calibration, 
accelerations, travel times (defined between two gates), and gap in seconds (33). 
DataFromSky’s capabilities with UAS video data show that the range of possibilities today 
using UAS for traffic monitoring is extensive. To understand how their methods would 
compare with the developed method, an analysis was completed, as described later in this 
section. 


1.1.5 Unmanned Aerial System Applications 


UAS, including large UAS, have historically been used for military applications. However, 
with the commercialization and reduction in cost and size of UAV in recent years, the 
potential uses for these devices has grown. UAS are comprised of three components: (1) the 
aircraft, or UAV; (2) communication and control; and (3) the pilot on the ground. There are 
two types of UAV: fixed wing and multirotor. Fixed wing UAV have an airplane-like design, 
generating lift from air passing underneath, and multirotor UAV have several rotors with 
propellers which push air downwards (34). While the fixed-wing design allows for a longer 
flight time, the aircraft must always be moving forward at a certain minimum speed to 
generate enough lift to stay in the air. Thus, it is not possible to keep the UAV hovering at a 
single location of interest. Further, fixed-wing aircraft require a large, open location for take- 
off and landing (34). Given these disadvantages, which do not affect multirotor aircraft, the 
multirotor aircraft is recommended to be used for data collection, especially for speed data 
collection. 


UAS applications have been explored for many uses, including for traffic monitoring, 
structural inspection, topographic surveying and mapping, and crash reconstruction (35). In a 


survey report by AASHTO in 2016, four specific benefits of UAS use were highlighted: 
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improved safety, time savings, decreased cost, and even decreased congestion, as there would 
no longer be a need to shut down lanes for stationary vehicles and machinery to complete 
tasks such as bridge inspections (2). 


1.1.5.1 Traffic Monitoring 

In recent years, UAS have been introduced to the transportation community as a cost- 
effective solution to collect trajectory data from the sky and replace the old approach of using 
pre-installed static cameras. Table 1.1, adapted from Barmpounakis et al., presents a 
subjective comparison of static camera use and UAS use for traffic monitoring and other 
related applications based on research found in previous literature. 


Table 1.1: Comparison between a static camera and an unmanned aerial system 


Metric Static Camera UAS 
Security/Privacy Medium Low 
Cost (acquiring and maintenance) Low Low 
Reusability Low High 
Energy efficiency Low High 
Deployment difficulty Low Low 
Operational time High Low 
Operation under adverse weather Medium Low 
Safety Risks Low Medium 
Endurance High Low 
Video post-processing skills required Medium High 
Data transfer, communication and storage Low High 
Operation skills required Low Medium 
Training requirement Low Medium 
Complexity Medium Medium 


Source: adapted from Barmpounakis et al. (6) 


Most multi-rotor UAS have the flexibility to collect large amounts of aerial data almost 
anywhere in a matter of minutes. Additionally, UAS can be programmed to automatically fly 
a particular route to collect specific aerial imagery, creating simplicity in the flying process 
for the pilot. Their small size is also beneficial to collect naturalistic data over a roadway, 
allowing for a more nonintrusive way of recording traffic data. However, a noteworthy 
limitation of multi-rotor UAS are their small battery capacities, which only allow them to fly 
for short periods of time; often for only 20 to 30 minutes (4, 7). However, provided that UAS 
can fly above a highway and collect the speeds of many vehicles at once, it is easily possible 
to collect more traffic data during that short amount of time than traditional methods. 
MassDOT procedures require that 100 vehicle speeds be collected during a weekday at off- 
peak hours at a singular location on a roadway for the speed limit—setting process, as 
mentioned above (1). Depending on the off-peak volume of the roadway of interest, this may 
not take UAS much time to collect data, especially as compared to manual collection by a 
RADAR or LiDAR gun. 


For example, on Thursday, November 17, 2016, count data was collected by MassDOT in 
Athol, Massachusetts, on the Mohawk Trail, a portion of Route 202 (36). From the period 
between 11 a.m. and 3 p.m., the average traffic volume was approximately 750 vehicles per 
hour. Assuming an ideal case of all vehicles traveling at free-flow speed, a UAS would only 
need to actively collect data in the sky for approximately eight minutes for 100 vehicle 
speeds to be collected. Even in a less than ideal case including fewer free-flow vehicles and 
assuming that only 50% of vehicles will be traveling at free-flow speed, the UAS would only 
need to collect data for 16 minutes. In the same case, but collected manually by an observer 
with a RADAR gun, the observer would have to visually see which vehicles are traveling at 
free-flow speed and directly record those vehicle’s speeds from the roadside. However, it 
would take more time, as if multiple vehicles traveling at free-flow speed were passing the 
observer at the same time (or even close to it), only one of those vehicle’s speed could be 
collected and recorded as the observer would need to use the gun to collect the data and then 
note it. Thus, this type of data collection would be much more time-intensive than through 
the use of a UAS. 


It is noted that for locations with really low volumes, data collection via UAS may not 
provide significant benefits over traditional roadside manual collection given that every 
vehicle would be traveling at free flow speeds and would not be obscuring other vehicles. 


Overall, it is possible that for many roadway situations, the short battery life of a multi-rotor 
UAS may not cause any issues. Extra batteries may also be carried if more than one 
deployment is necessary. Another issue may be that, according to FAA regulations Part 107, 
UAV can only be flown in fair wind and weather conditions; this limits their use. These 
weather conditions are also necessary to collect accurate data from a UAS, given that wind 
and other weather conditions can cause the camera connected to the UAV to shake. Since 
MassDOT procedures require that data be collected under ideal weather conditions for the 
speed setting process, this should not be an issue for the data collection (1). Finally, it is 
noted that UAS must be operated by a Part 107 certificate holder, if flown for non- 
recreational uses. This certificate requires a written exam to be passed, which costs $75 and 
must be retaken every two years to maintain certification. 


Studies were completed using UAS for traffic surveillance, as well as roadway incident 
monitoring, as discussed by Lee et al. and Wang et al. (4, 5). When utilizing UAS for traffic 
monitoring, it is important to consider data collection resolution and ground sampling 
distance (GSD), to therefore consider accuracy error. The most basic parameter of resolution 
is the number of pixels of the recorded area; as pixels increase at a given altitude of 
recording, and therefore, at a given fixed sensor width that the camera lens is covering, 
resolution increases (3). This is due pixel size; the less unit area a pixel covers, the higher 
resolution the image is. Decreasing the unit area covered by a pixel can also be changed by 
decreasing the altitude of where the image is taken. In a study completed in 2016 by Wang et 
al., vehicle detection for traffic monitoring was found to be most accurate when the altitude 
of the UAV (a DJI Phantom 2 in this study) was within the range of 100 meters (328 feet) to 
120 meters (393 feet), covering a road section length of 190 meters (623 feet) and 228 meters 
(722 feet), respectively. In this study, the camera utilized had a resolution of 1920 x 1080 
pixels, and horizontal field of view of 94.4 degrees, vertical field of view of 55.0 degrees, 
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and diagonal field of view of 107.1 degrees. It is noted that the video recording was done ina 
fashion that the vehicles traveling were positioned horizontally. Further, when the altitude 
increased from 120 meters to 150 meters (492 feet), the accuracy of tracking decreased from 
approximately 99.8% to 96.1% (4). Per FAA Part 107 regulations, UAV may not operate 400 
feet above the highest structure in its vicinity, so flying below 400 feet when recording is 
optimal in this regard. Wang et al. utilized a particular method to find these accuracy results; 
this method jointly utilized three image features to detect and track vehicles: edge, optical 
flow, and local feature point (4). This specific method was designed for vehicle detection and 
tracking to improve efficiency and accuracy. As discussed previously, video stabilization 
applications can increase accuracy of the tracking data (3). 


The ground sampling distance presents the spatial resolution. In short, the GSD is the true 
distance between two consecutive pixel centers measured on the ground. The higher the 
GSD, the lower the spatial resolution, and vice versa. GSD is calculated by dividing the size 
of a pixel (mm) on the sensor by the focal length (mm), and multiplying by the distance from 
the camera to the ground (cm) (37). 


1.1.5.2 Commercial Applications 

UAS are increasingly being employed for a number of applications outside of traffic 
monitoring, within and outside of the field of transportation. Given the large cost savings that 
is possible with using UAS over manual work, along with improved safety, time saving, and 
a reduced need for lane closures (if transportation-related work), they have been deemed as 
highly beneficial for industry tasks and projects by AASHTO. According to a survey report 
from AASHTO in 2016, bridge inspection costs can be saved when using UAV over manual 
inspections. It was estimated that over $4,000 could be saved during a bridge deck inspection 
using the technology (2). Additionally, UAS imagery has been found to be superior to 
conventional aerial photography because the camera on the UAV can be closer to the subject. 
This can be useful for surveying large areas, roadway mapping, and crash reconstruction (7). 
It is estimated that using a UAS to document a crash scene decreases the time spent on the 
roadway by 80% and the time spent taking measurements by 65% compared to traditional 
methods (7). In 2013, the Traffic Support Unit in the Highway Safety Division in Ontario 
mapped major collision scenes in just 22 minutes, on average, using UAS (7). This increases 
safety of first responders, reduces the economic impact on drivers from lost time, and 
increases the safety of the roadway through the reduction of possible secondary collisions. 


1.2 Literature Summary 


UAS are already being utilized in numerous transportation applications and have been proven 
to save time and cost in those efforts. In the development of this research of UAS for vehicle 
tracking purposes, the existing literature has and will continue to inform its direction. As an 
example, the research on operating characteristics of UAV (battery life, accuracy of vehicle 
tracking at various altitudes, etc.) and camera parameters (resolution, field of views) will be 
helpful in determining flight parameters. Additionally, the background literature with respect 
to speed studies is critical in determining the sample sizes of data needed, and thus flight 
durations. This is necessary to understand in order to gather a sufficient dataset for the speed 
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limit setting process. Further, existing methods for object tracking, as outlined in the 
literature, will be tested first for data post-processing to determine the most accurate and 
efficient way to obtain vehicle data from the video. Finally, it is clear that there is a strong 
need for an improved methodology to collect operating speeds and that UAS offer a 
promising solution to meet those needs. 
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2.0 Research Methodology 


This study investigated two primary applications of UAS: the capture and analysis of vehicle 
speeds and vehicle volumes. The methods employed for the two applications are documented 
herein. 


2.1 Volume 


Traffic volumes play a crucial role in decisions related to planning, design, operation, and 
management of roadways. Traditionally, short term traffic volume is collected through 
intrusive pneumatic tubes or manual counts. This sub-section further describes using UAS 
and vehicle tracking to collect this data. 


Multiple intersections were considered for initial data collection to determine which location 
would be most appropriate for further analyses. These locations included the roundabout 
intersecting North Pleasant Street, Eastman Lane, and Governors Drive on the University of 
Massachusetts Amherst campus, the roundabout in downtown Amherst intersecting East 
Pleasant Street and Triangle Street, and the double roundabouts at Atkins Corner in Amherst. 
Images taken from the UAS at each of these locations are shown in Figures 2.1, 2.2, and 2.3. 


Figure 2.1: North Pleasant Street/Eastman Lane/Governors Drive roundabout 
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Figure 1.2: East Pleasant Street/Triangle Street roundabout 
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Figure 2.2: Double roundabout at Atkins Corner 


After trial flights, the location at Atkins Corner in Amherst was chosen due to its complexity. 
It was considered a location that would benefit most from UAS vehicle tracking as compared 
to traditional methods of vehicle counting. 


The drone used in this study was a DJI Phantom 3 Pro. This drone’s camera has a FOV of 94 
degrees and was flown at varying heights below the maximum of 400 feet. The altitude of 
each flight was chosen based on the minimum height to capture the full intersection. For the 
chosen double roundabout intersection, the drone was flown at 400 feet above ground level. 


14 


As the area of the required study area increased, the required drone altitude increased; in 
turn, resolution decreased. Given the specifications of the built-in standard camera on the 
Phantom 3 Pro, the GSD at a 400 feet (122 meter) flight was calculated to be 10.84 
centimeters/pixel (nadir-pointing). 


To obtain the most realistic results, twelve videos averaging seven minutes each were taken 
of the double roundabout during the morning peak hour from 7 a.m. to 9 a.m. as this timeset 
is most typically used for transportation planning. Each video was manually counted for 
vehicles to establish a ground truth by which the accuracy of the automated counting could 
be compared. 


2.1.1 Automated Vehicle Tracking Using Computer Vision 


The research team developed an automated vehicle tracking method using computer vision in 
this study by employing two primary steps, which included a You Only Look Once (YOLO)- 
based vehicle detection model, and a Kalman filter-based vehicle tracking model. Figure 2.4 
shows the flowchart of the proposed method. 


STEP 1 STEP 2 STEP 3 


Video Preprocessing Vehicle Detection Vehicle Tracking 


Figure 2.3: Flowchart of proposed method for vehicle tracking 


STEP 1 Video Preprocessing: The objective of this step is to effectively downsample the 
video frames and the image resolution for each frame. An effective downsampling strategy 
could preserve all the necessary features in the image and the frame rate for the subsequent 
detection and tracking steps, respectively, while minimizing processing time. The 
downsampling of the video data was based on a frame downsample by a factor of 15, i.e., 
resulting in 2 fps, and a resolution downsample by a factor of 2 in both x and y-direction, i.e., 
resulting 1920 x 1080. The frame downsampling factor and the resolution downsampling 
factor were iteratively determined using a subset of the collected data by evaluating the 
corresponding detector rate and processing time. 


STEP 2 Vehicle Detection: The objective of this step is to identify the location of the 
vehicle appearing in each image frame. In this study, a deep learning framework called 
YOLO (38, 39) was employed for identifying the vehicles. The outcome of this method will 
present the bounding box of the detected vehicle in the image coordinate system. Figure 6 
shows an example of the detected vehicle. YOLO has been widely applied in vehicle 
detection using pre-trained models from datasets such as Microsoft Common Object in 
Context (COCO). However, very few models have been trained from UAV videos, which 
show different outlooks and features for captured vehicles as compared to more traditional 
datasets. 
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Therefore, the research team retrained a new model for the images captured from UAV based 
on an open vehicle dataset collected by Kharuzhy (40). The research team tuned the 
following hyperparameters that are associated with the efficiency of the training process, 
including learning rate (i.e., learning rate schedule and decay), batch size, momentum, and 
decay. 

e Learning Rate: The learning rate is defined as the step size of the gradient descent 
flow during the training process. Generally, a large learning rate may cause the model 
to converge rapidly to a suboptimal solution, while a small learning rate may cause 
the training process unnecessarily long. The learning rates need to be dynamically 
configured to balance the accuracy of the model and the efficiency of the training, 
through learning rate schedule and decay. 

oO Learning Rate Schedule: In this study, the learning rate is increased from 
0.001 to 0.01 along with the epoch reaches 75th iteration, to ensure that the 
model can be training faster after the majority of the iterations has been 
completed, when it becomes less risky to be trapped at a suboptimal solution. 

oO Decay: Aside from the abrupt changes in the learning rate as defined in the 
learning rate schedule, the decay defines the gradual changes of learning rate 
in consecutive iterations so that the learning rate is continuously decreasing as 
the training process advances further. In this study, the decay value of 0.00005 
is used as a default. 

e Batch Size: The batch size defines the number of training samples that will be 
propagated into the network for each iteration. Generally, a larger batch size will 
provide a better estimate of the gradient for the training process, so that the training 
will be rapidly converged (i.e., faster identification of the gradient descent). However, 
due to the limitation of the computation resources, it is not always feasible to use a 
large batch size. In this study, a batch size of 64 is used to balance the performance 
and the constraints of the computation resources (i.e., Nvidia GTX 1080 Ti in this 
study). 

e Momentum: The Momentum refers to a method that facilitates the gradient flow 
optimization in the most relevant direction by dampening the strength of the 
irrelevant directions (41). The momentum value is defined as the magnitude of such a 
dampening effect. A larger momentum would enable a much faster convergence be 
limiting the oscillation effect during the gradient descent in training. In this study, a 
momentum value of 0.9 is used to enable faster convergence without significantly 
suppressing the possible change of direction change in the gradient descent process. 


While the research team did not find significant accuracy differences by adjusting these 
parameters, it did identify that a more aggressive learning schedule with larger decaying 
values may significantly reduce the training time based on the limited number of UAV data 
in this study. In the processing of detection, once the model is trained, the detection process 
requires minimum manual intervention. However, the confidence level for the detected 
candidates needed to be “empirically” determined to optimize the precision-recall curve. In 
this study, a dataset containing 3,645 vehicles were used for determining the confidence 
level. Figure 2.5 shows the precision-recall curve. Based on the derived precision-recall 
curve, a confidence level of 0.5 was selected for the detection algorithm in this study, as it 
has the best balance between false positives and false negatives. 
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Figure 2.4; Precision-recall curve to determine confidence level 


describe pixel size of objects being tracked. 


Figure 2.5: Example of the detected vehicles 


STEP 3 Vehicle Tracking: The objective of this step is to associate the detected location of 
the vehicles (i.e., on the detector) from consecutive frames into the same track, so that the 
subsequent vehicle counting and speed computation become feasible. In this study, the 
detector is represented by the centroid of the bounding box from STEP 2. Figure 2.7 shows 
an example of a tracked vehicle superimposed on the captured frame. In this study, the 
Kalman filter (42) was employed to predict the motion of each detector. Based on the 
closeness of the predicted location and the observed location (i.e., the detector from the 
following frames), the current detector will be merged to the vehicle track or split into a new 
track (43). Although the research team attempted to achieve the best detection rates in STEP 
2, the vehicle tracking algorithm was able to correct a limited number of false positives and 
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false negatives when these cases did not propagate in many consecutive frames. In this study, 
the tracking strategy allows a trajectory removal if the detector appears in less than five 
frames, and allows a trajectory splitting if the detector no longer appears in more than ten 
frames. 


Vehicle Track #8 - Image Framess #1010 - #1692 


: ss 


Note: Red line depicts the tracking of a vehicle that entered from botto 


@ 3% 


m R and exited to top L. 


Figure 2.6: Example of vehicle tracking 


2.2 Speed 


As described throughout the literature review, speed data is needed in the speed limit setting 
process, among other transportation studies. This sub-section describes the methods of 
collecting speed data using a UAS and vehicle tracking methods. 


The same DJI Phantom 3 Pro drone was utilized for the speed experiments, and included the 
same camera specifications, a FOV of 94 degrees. Originally, this experiment was going to 
take place along a portion of South East Street in Amherst. However, it was found that we 
needed ground measurements to determine how much distance was represented by a single 
pixel. Through another research project, we already knew lane widths, shoulder widths, etc. 
for Route 9 in Amherst, so we switched our experiment to fly at that location. An image of 
this location is shown in Figure 2.8. 
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Note: Bottom-most vehicle was the probe vehicle and is indicated by blue “X.” 


Figure 2.7: Video still from Route 9 speed validation location 


To verify the accuracy of the speed data, probe drives were conducted simultaneously during 
video data collection. This was done by placing an “X” on the top of a vehicle and traversing 
the length of the roadway in the drone’s view, while the drone flew at an altitude of 100 
meters (328 feet). The probe vehicle drove at various speeds and tracked their speed using 
both their speedometer and a smartphone app. Given the specifications of the built-in 
standard camera on the Phantom 3 Pro, the GSD at a 100 meter flight was calculated to be 
8.89 centimeters/pixel (nadir-pointing). 


2.2.1 Automated Speed Processing 


In this study, the automated speed computation is based on the vehicle tracking results 
presented later in Section 3.1. The outcome of the vehicle tracking is a trajectory that is 
represented in the image coordinate system. In other words, the location of the vehicle within 
the trajectory is digitized based on the x, y coordinate of the image. To compute vehicle 
speed, the real-world representation of the locations with geometrical information is required. 
Therefore, in this study, the research team extended the vehicle tracking flowchart with two 
additional steps for speed computation. Figure 2.9 shows the updated flowchart for speed 
processing. 
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STEP 5 
Speed Computation 


STEP 4 


Camera Calibration 


Figure 2.8: Flowchart with two additional steps for speed computation (in bold) and three steps 
for target tracking 


STEP 4 Camera Calibration: The objective of this step is to transform the image coordinate 
system to the world coordinate system so that the distance measured in the unit of the pixel 
can be translated in the unit of feet or miles. In this study, the research team developed a 
simple homography for transforming the coordinate pair, which is a 3x3 matrix. As the 
homography has eight degrees of freedom, at least four-point pairs are required for 
computing the homography matrix (44). Figure 2.10 illustrates an example of the 
transformation where the four points shown on the left are the ones represented in the image 
coordinate system, while the matching four points shown on the right are the ones 
represented in the world coordinate system. 


} ‘i 


Note: Green box in L image is transformed to rectangle in R image from aerial perspective. 


Figure 2.10: Example of transformation 


STEP 5 Speed Computation: The objective of this step is to compute the vehicle speed for all 
extracted vehicle trajectories. In this study, the research team first computed the speed based 
on the distance measured in the world coordinate system in consecutive frames and divided 
the time (i.e., frame rate) between those consecutive frames. As presented in STEP 1 in 
Section 2.1.1, the original video was downsampled in frame rate by a factor of 15 accounting 
for the fact that the denominator of the speed computation in this step is 0.5 seconds. In any 
speed computation scenario, the accuracy of the vehicle localization (i.e., STEPs 2 and 3) and 
the accuracy of the camera calibration (i.e., STEP 4) may affect the accuracy of the speed 
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computation. The accuracy of the camera calibration can be easily maintained thanks to the 
consistency of the geometry measures, and the impact of the possible error source (i.e., 
quantization) is only negligible. However, the accuracy of the vehicle localization is 
challenging to maintain, as the vehicles are represented by the centroid of the detected 
bounding boxes. The accuracy for the detected/tracked centroids to reflect the vehicle’s 
actual location are prone to the change of view angle, partial occlusion, or imperfect 
detection, which are inevitable. The slight shift of the centroids in consecutive frames may 
create the locational disturbance. In this study, the data collected by UAV have the advantage 
of avoiding a drastic change of view angle and occlusions. Recognizing that the imperfect 
detection will persist in any vehicle detection and tracking algorithms but can be minimized 
(i.e., STEPs 2 and 3), the research team applied a simple median smoothing scheme on the 
derived speed with a window size of five, so that the randomly-occurred, imperfect detection 
(i.e., incorrect centroid) will be effectively disregarded. Figure 3.5 and Figure 3.6 show some 
of the examples before (i.e., blue lines) and after (i.e., orange lines) the filtering. 
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3.0 Results 


3.1 Volume 


The following sub-section describes the accuracy and cost of the methods used, as well as a 
comparison between commercial products and traditional methods. It is noted that while this 
analysis only explored an origin-destination study, it could also be used for a turning 
movement count (TMC) This could be explored in potential future work. 


3.1.1 Accuracy of Automated Tracking 


The recall of each video was calculated. The recall was calculated as the true positives 
divided by the true positives plus the false negatives. In other words, the recall was the actual 
number of vehicles that passed by that the software did pick up, divided by the total number 
of vehicles that actually did pass through the video. Thus, this recall is a good measure of 
accuracy. Across all of the videos analyzed, the recall was 93%. It is noted that the full tables 
of the ground truth, true positives, and false positives are included in Appendix B—Data 
Analysis Tables. 


The recall was separated by origin-destination. The labels for these are shown in Figure 3.1 
and broken down into the following: 

e TL (top left): From or to Bay Road, west side 

e BL (bottom left): From or to Route 116, south side 

e TR (top right): From or to Route 116, north side 

e BR (bottom right): From or to Bay Road, east side 


Figure 9: Origin-destination labels 
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Table 3.1 presents the recall for each origin-destination in the video. It is noted that N/A 
represents an origin-destination pair in the video where no vehicles were present. Each origin 
had three possible destination pairs, for a total of 12 pairs. The U-turn (ex. BL-BL) origin- 
destination pairs were omitted as the traffic volume was near zero for each one. 


Table 3.1: Recall of each origin-destination pair in each video 


Timestamp | TL-TR | TL-BL_ TL-BR | TR-TL | TR-BL | TR-BR_ BL-TL | BL-TR | BL-BR | BR-TL | BR-TR'  BR-BL 
7:00:00 100% | 64% | 80% 100% 68% | N/A 67% 80% | N/A | 76% N/A 71% 
7:09:20 N/A | 80% | 100% N/A | 90% | N/A | 76% | 75% | 100% | 86% N/A 100% 
7:19:00 100% | 75% | 85% 100% 92% | N/A 93% 96% | 100% | 94% N/A 89% 
7:28:20 100% | 100% | 100% 100% 90% | N/A | 96% 89% | 100% | 88% 100% 89% 
7:41:05 100% | 100% 100% 83% 100% | N/A 94% 100% | 86% | 90% N/A 89% 
7:50:26 100% 100% 100% | 100% | 94% 100% 88% 92% 100% 95% 100% | 100% 
8:00:00 80% | 92% 87% 100% 89% | 100% | 92% | 90% | 100% 100% 100% 86% 
8:09:20 100% | 100% 100% 83% 100% | 100% 83% 94% | 100% 100% N/A 100% 
8:21:00 100% 93% | 95% | 100% | 88% | 100% 85% 90% 100% 87% | 75% 91% 
8:30:20 86% | 90% 100% 100% 94% | 100% | 83% 90% | 83% 87% | 100% | 100% 
8:47:41 100% | 100% 91% 100% 93% | N/A | 97% | 89% | 80% | 88% 100% 86% 
8:57:14 100% | 100% 100% N/A 80% | N/A | 86% | 89% | 100% | 95% 100% 100% 


Precision, which are the true positives divided by the total of the true positives plus the false 
positives, averaged 93% as well. This is presented in Table 3.2. 


Table 3.2: Precision of each origin-destination pair in each video 


Timestamp | TL-TR | TL-BL_ TL-BR | TR-TL | TR-BL | TR-BR_ BL-TL_ BL-TR| BL-BR_ BR-TL | BR-TR_| BR-BL 
7:00:00 100% | 70% | 80% 100% 87% | N/A | 73% | 76% | N/A | 86% | N/A 83% 
7:09:20 N/A | 80% | 100% N/A | 90% | N/A | 81% | 80% | 100% 86% | N/A 100% 
7:19:00 83% 100% | 85% 100% | 92% N/A 93% 92% | 92% 86% | N/A 100% 
7:28:20 100% | 100% | 89% 100% 100% | N/A 92% | 86% 100% 88% | 100% 89% 
7:41:05 100% | 100% 88% 100% 86% | N/A | 88% | 96% 100% 96% | N/A 100% 
7:50:26 80% | 90% 100% 100% 84% | 100% | 94% 96% 100% | 90% | 100% 88% 
8:00:00 100% 92% | 87% | 100% | 89% | 100% 85% | 86% 100% | 93% 100% 86% 
8:09:20 100% | 100% 100% 83% 100% | 100% 100% | 86% 100% 92% | N/A 100% 
8:21:00 86% | 100% 90% 100% 88% | 100% | 89% 97% | 100% | 87% | 100% 100% 
8:30:20 86% 90% 100% 100% 94% | 100% | 88% 87% | 100% | 100% | 100% 80% 
8:47:41 100% | 84% | 91% 100% 83% | N/A | 92% | 82% | 80% 93% | 100% 92% 
8:57:14 100% | 75% | 88% N/A | 100% | N/A | 86% | 89% 100% 90% | 100% | 100% 


3.1.1.1 Impact of Lighting on Detection 

In the early morning, specifically in videos captured from 7 a.m to 7:20 a.m., accuracy was 

lower than the rest of the morning. This was due to the poorer lighting in the early morning, 
as the sun was still rising. This is represented in Figure 3.2 and Figure 3.3, which show the 

UAS captured images at 7 a.m. and 8:30 a.m., respectively. 
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Figure 11: Atkins Corner at 8:30 am depicting light conditions 


To overcome this issue with low-lighting, which interferes with detection, other detection 
methods could be explored. This is an important barrier to overcome, as traffic data during 
the peak hours from 7 a.m. to 9 a.m. and 4 p.m. to 6 p.m. is typical, and these times coincide 
with darkness during the winter months. 
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3.1.2 Processing Time 


The processing was based on a frame rate of two frames per second and resolution of 1920 x 
1080 pixels. In total, for 1.47 hours of video, the processing time for detection was 1.74 
hours, for tracking was 0.84 hours, and for the auxiliary time was 0.12 hours. In total, 2.71 
hours were required in automatic processing time for 1.47 hours of videos. Thus, for every 
one hour of video, approximately 1.8 hours of automatic processing was required. The details 
of the processing time for each video is provided in more detail in Table 3.3. 


Table 3.3: Processing time for each video 


Video Length Automatic 

, ; Manual : F as 

Timestamp Duration Detection Tracking Auxiliary Total 
Frame (sec.) 

(sec.) (sec.) (sec.) (sec.) (sec.) 
7:00:00 560.2 16788 2400.0 698.4 324.7 50.7 1073.8 
7:09:20 274.3 8221 1500.0 296.0 168.7 25.8 490.5 
7:19:00 560.4 16795 3000.0 678.5 366.4 55.0 1099.9 
7:28:20 465.6 13954 2580.0 558.2 320.9 45.7 924.8 
7:41:05 561.8 16838 3300.0 619.6 294.3 41.5 955.5 
7:50:26 382.1 11451 2100.0 490.1 264.7 39.7 794.5 
8:00:00 560.7 16805 4200.0 638.6 265.0 42.1 945.7 
8:09:20 285.0 8542 1920.0 328.0 195.2 26.3 549.5 
8:21:00 560.8 16806 3300.0 645.4 271.0 36.6 953.0 
8:30:20 346.8 10395 2400.0 440.7 196.1 27.1 663.9 
8:47:41 562.8 16867 4200.0 674.7 269.9 37.2 981.8 
8:57:14 167.2 5010 900.0 210.4 84.2 13.5 308.1 


3.1.3 Comparison of Existing Methods 


As previously mentioned, one commercial product that is able to analyze video data to 
extract vehicular data is DataFromSky. To compare how our analysis technique differs from 
theirs, a portion of the same video collected via UAS for our analysis was analyzed using one 
of their services, DataFromSky Light. This is a fully automatic traffic data extraction 
software from DataFromSky and offers a license-free version of their software and has most 
of the licensed DataFromSky full version functionalities, including traffic counting, headway 
statistics computation, trajectories visualization, and export of trajectories in various formats 
(45). 


For the use of this software version, DataFromSky recommends a high video resolution and 
object tracking size, among other specifications, though they state the service can provide 
accurate results even beyond these parameters. To use this free service, a video under 2GB 
was uploaded to a website. To achieve a size under 2GB, an approximate three-minute video 
was uploaded from a portion of the 7 a.m. video in our analysis. This video was chosen as it 
had the worst accuracy with a recall of 78% on average, and we wanted to explore if this 
method would be more accurate. The license-free software version was then downloaded, 
and after approximately four hours, the analysis was complete by DataFromSky Light. The 
viewer of this software is shown in Figure 3.4. 
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Figure 12: Screenshot from DataFromSky video analysis program 


In the three-minute video, a total of 15 vehicles were not detected, 18 vehicles had partial 
detection, and only two vehicles achieved full detection throughout their whole trip. The 
video misclassified cars as motorcycles 16% of the time. On their website for the full 
version, DataFromSky Light states that they can provide a guaranteed hit rate of over 96% 
(45). Using only the free version as a comparison, the accuracy of vehicle detection was 57% 
for at least partial detection, and only 5.7% for full detection. Again, through our analyses of 
the same video, the lowest detection was 78%. 


Using traditional methods, the accuracy ranges from 96% for pneumatic tubes to manual 
counting at approximately 100% (46). 


3.1.4 Cost Comparison 


In order to completely assess the viability of UAS as a substitute, or supplement, to existing 
traffic volume techniques, we completed a cost comparison. 


3.1.4.1 UAS with Automated Tracking 

The monetary cost of this collection method would include the UAS itself, which in the case 
of this project, was the DJI Phantom 3 Pro drone. This drone currently retails for 
approximately $1,000 or less. An additional cost of this technique would be a drone-certified 
pilot. A current drone license requires an exam to be taken, which costs $75 and needs to be 
re-taken every two years. This exam can be taken at most airports with prior scheduling. The 
time-cost of this developed method, as previously mentioned, would be the actual collection 
time, set up time of the vehicle tracking software, and the processing time. While the drone 
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flight collection time may vary, this project had a processing time of 1.8 hours of automatic 
processing for each one hour of video. 


Another option to obtain video from a UAS without purchasing one or flying it, would be to 
hire a pilot. UMassAir offers flight services; an informal quote was obtained and stated to 
cost $100 per hour. For this project, with two hours of data collected, it would cost $200 for 
UMassAir to collect the video. 


3.1.4.2 UAS with Existing Software 

For the commercial product of DataFromSky, the same drone flight to collect the data would 
be needed and the only difference would be its processing. Using DataFromSky Light, which 
is free to use, processing a three hour video would take more time, as only 2GB of video data 
(equating to approximately three minutes of video) can be uploaded at once. It took 
approximately four hours for the software to send back the results. 


One of the paid versions of DataFromSky is the DataFromSky Viewer. This cost per year per 
license was €1150 ($1,292 US as of June 2019) for the basic version, which includes 
counting at specific points, turning movement counts, and speed data, among other features. 
For their traffic analysis service, multiple fees for different aspects of the video are added to 
get a price. For example, for one hour of video of a traffic flow between 1,500 and 4,500 
vehicles per hour, the cost would be €465 ($522 US as of June 2019). Each of these cost 
breakdowns can be found in Appendix C—Quoted Cost Sheets. 


3.1.4.3 Traditional Methods 

Traditional methods to collect volume data at the chosen location of the double roundabout 
was also analyzed. Innovative Data, Inc. is one company that completes this type of data 
collection. A quote of this location for a 90 minute collection with data input into Excel came 
to $3,190. This quote can be found in Appendix C. 


For a simple four-way intersection where a TMC would be appropriate, the cost per hour for 
manual collection by a technician would be $55. Thus, compared to the UMassAir flight cost 
per hour of $100, it would more cost effective to complete a traditional TMC by hiring a 
technician using a count board. However, if the intersection is more complex and would 
require two or more technicians at a location at one time, it would likely be more cost 
effective to use a UAS and the developed vehicle tracking method. 


3.1.4.4 Cost Comparison Summary 
A comparison of the various methods for this study is summarized in Table 3.4. 
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Table 3.4: Cost comparison of methods 


UAS Developed Method DataFromSky ianouatie 
Cost ’ UMassAir : . DataFrom 
In House Collection Collection Paid Version Sky Light Data, Inc. 
One Time Cost $1,075 to $2,075 SO SO SO SO 
Per Use Cost SO $200 $385.50 SO $3,190 
4 hours/3 
Processing Time-cost 2.71 hours 2.71 hours Unknown min of 0 hours 
video 
Accuracy High High Unknown Low High 


3.2 Speed 


The following sub-section describes the outcomes of speed detection using computer vision, 
as well as its accuracy and cost compared to other methods. 


3.2.1 Accuracy 


The accuracy of the automated drives compared to the probe drives is presented in Table 3.5. 


Table 3.5: Accuracy of probe drives for southbound (SB) and northbound drives (NB) 


Drive , : Actual Speed Average Measured Speed Relative 
Direction 

Number (mph) (mph) Error 
1A SB 45 48.8 8.4% 
2A NB 44 47.5 8.0% 
1B SB 50 52.9 5.8% 
2B NB 51 52.4 2.8% 
1C SB 55 59.5 8.3% 
2C NB 54 57.3 6.2% 
Average: 6.6% 


The relative error values ranged from 2.8% to 8.4%, with an average of 6.6% across all of the 
drives. This error was larger than anticipated and shows that further research must be 
completed to achieve better accuracy. The measured speeds across the roadway section 
during its length are presented in Figure 3.5 and Figure 3.6. Overall, given the lower 
accuracy of the data than anticipated, it is noted that this data would not yet be an accurate 
representation of the space-mean speed, as mentioned in the Literature Review section. 
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Figure 13: Southbound drive speeds 
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Note: Blue ‘x’s’ are individual measurements; red line is a best fit of the individual points; and green line is the 
ground truth speed. X-axis represents time passed by frame and y-axis represents vehicle speed (mph). 


Figure 14: Northbound drive speeds 


The partial version of DataFromSky does not provide speed in kilometers or miles per hour; 
thus, a comparison only with DataFromSky’s vehicle counting ability was able to be made, 
as presented in the previous section. The full version of DataFromSky has the additional 
capabilities of extracting detailed speed and acceleration data, lane utilization, and average 
speeds. These details are shown in Appendix C. However, a speed comparison was unable to 
be completed. 
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Traditional methods of speed data collection on a short term basis are pneumatic tubes, 
LiDAR, or radar. Pneumatic tubes were found to have a speed error of 4.2%, or 
approximately between -2 mph and +2 mph at an actual speed of 45 mph (47). LiDAR and 
radar sensors have accuracies between -1 mph and +3 mph (48, 49). 


Comparing an average error of 6.6% from the probe drives to 4.2% from pneumatic tubes, it 
is clear that the new data collection method for speed may not yet be on par with traditional 
methods, but is also not that far off. The vibrations of speed computation are mainly due to 
the speed being computed based on the differential of distance measured frame by frame. 
Any quantization error in the vehicle detection and in the camera calibration, mainly due to 
the difficulty of digitizing the road edges consistently and assuming the road width remains 
the same, may introduce a small disturbance in distance measurement. Along with small 
viabrations attributed to the hovering UAS, this results in vibrations in speed. Future testing 
and work is recommended to further study different methods to gain a better accuracy that is 
closer to the error, or better, than traditional methods. 


3.2.2 Cost 


The cost of speed data collection using our developed method is similar to the cost of 
collecting volume data, as the data collection time and cost is the same. This would include 
the drone cost (approximately $1000 to $2000 for the DJI Phantom 3 Pro) and time-cost of 
collecting the data in the field through a flight. 


The time-cost of some traditional methods, such as LiDAR and radar, are larger than using a 
UAS in the field, as UAS are able to collect data from almost all of the vehicles passing by, 
rather than just the ones that a data collector chooses to point their sensor at in the field. 
Further, processing time for the collected LiDAR and radar data exists as well, as it must be 
input into a computer and analyzed manually or through another program. This differs from 
pneumatic tubes, which are able to collect data from every passing vehicle. However, the 
time-cost of pneumatic tubes is also time-consuming, as they require time to be installed into 
the pavement and removed, as well as processing time of the data after extraction, especially 
at more challenging locations with higher traffic volumes. As an example, a time quote 
provided by Innovative Data, LLC stated that at more challenging locations, tube and 
equipment preparation would take 0.5 hours, field installation and removal would take 0.5 
hours per installation site, and downloading and processing the data would take 0.25 hours. 
Moreso, at some very high volume locations, such as a state highway, police detail would be 
required to install and remove these tubes to create a safer environment for the technicians 
working on the roadway. Thus, at high volume or generally unsafe locations to install 
pneumatic tubes, UAS data collection would be most useful. 
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4.0 Implementation and Technology Transfer 


This research project made great progress in developing a method to capture traffic speeds 
and volumes using a small unmanned aerial vehicle. In order to achieve successful 
implementation of the developed methods, future work is needed. The algorithms and 
processes used to automate the analyses currently require a deep understanding of image 
processing and a strong proficiency in an engineering language, such as Matlab or Python. 


Future work should focus on training a diverse multitude of models so that the computer 
vision processing can be applied to any UAS video with minimal human intervention. These 
models would then need to be incorporated into a software program with a graphical user 
interface (GUI). Such a program and interface would enable any user to process their own 
UAS videos with minimal training and expertise. 


In the meantime, UAS could be used for niche applications with manual processing of the 


resultant videos. Such applications would include origin-destination studies at large rotaries 
or adjacent roundabouts, such as the one analyzed in this study. 
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5.0 Conclusions 


This project explored the use of UAS as a traffic data collection tool as compared to 
traditional speed data collection instruments on roadways through a field study. Previous 
literature has found that UAS are being utilized for survey work and are new devices used in 
the field of traffic data collection. Using the data collected in the field and a literature review, 
a methodology was developed to understand the accuracy of the data and how it may be 
useful in the speed-limit setting process. Two studies were completed to understand the 
accuracy and cost of using UAS as compared to traditional methods: one for volume data 
collection and another for speed data collection. Using a UAS and video processing, our 
method was found to have a count accuracy of 93% on average. Further, the speed data 
collection using our developed method had a relative error on average of 6.6%. For the 
speed-limit setting process, more detailed speed data may be required. However, it is noted 
that the speeds collected through our method were in the same range as the errors 
experienced through the use of LiDAR and radar sensors, which are traditionally used today. 
These sensors have a range of error from -1 mph to +3 mph. 


Compared to traditional methods, our developed method has a similar upfront monetary cost 
of equipment. The large difference between the methods is the time-cost. One UAS flight is 
able to capture all of the vehicles passing through a location through the use of a video; 
however, LiDAR and radar sensors, when used manually, only collect one vehicle’s data at a 
time. Thus, UAS on medium to high volume roadways have the potential to be more time- 
cost effective than traditional methods. 


Future work to further develop the use of UAS for traffic monitoring could include turning 
movement counts, conflict-event studies, intersection delay measurements, parking 
utilization tracking, and queue studies. Additionally, further research work could continue 
exploring the best vehicle tracking methods using UAS to gain the most accurate results, 
including researching the use of a thermal camera instead of a standard camera in dark 
lighting. Thermal cameras are already being used on UAS in other industries (50). Finally, 
these future results may include volume and speed studies along with the development of a 
software platform to aid with the implementation of UAS for traffic data collection. 
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7.0 Appendices 


7.1 Appendix A: Traditional Speed 
Collection Techniques 


7.1.1 Intrusive Devices 


Currently, there are five types of intrusive speed collection devices that are common to use 
today: inductive loops, pneumatic tubes with Automatic Traffic Recorders (ATRs), 
piezoelectric sensors, bending plates, and magnetic detectors (23). Each of these devices are 
analyzed in this section. 


7.1.1.1 Inductive Loops 

Inductive loops are wire loops that are installed in the pavement of a roadway to detect 
various traffic data, including speed data (25). However, to collect speed data, multiple 
detectors are often required. Typically, these devices are utilized to collect long-term traffic 
data at a location, as they need to be installed into the roadway using saws, which may cause 
disruption of traffic for installation and repair (25). 


7.1.1.2 Pneumatic Tubes 

Pneumatic road tubes with ATRs are rubber tubes that are placed across a roadway to detect 
vehicles through pressure changes that are produced as vehicles pass over the tubes. While 
they are accurate at collecting speed data (51), they require labor to be installed and 
uninstalled at specific locations on a roadway. This requires a temporary interruption of 
flowing traffic at many locations, depending on the time of installation and roadway 
volumes. During this installation and uninstallation, workers must cross the roadway, which 
creates a safety hazard. In some circumstances, police are required to be present throughout 
this process to provide the safest environment for roadway users and the installation workers. 
This type of data collector differs from many of the other traditional intrusive and 
nonintrusive devices, as they are not permanent and are meant to be placed at a specific 
location for only a short time. 


7.1.1.3 Piezoelectric Sensors 

Piezoelectric sensors gather traffic information, such as speed, by detecting the passing of a 
tire. Similarly to inductive loops, they also require traffic disruption for installation and 
repair, making them more useful for long-term data collection locations (25). 


7.1.1.4 Bending Plates 

Bending plate systems record the strain as a vehicle passes over it and calibrates to calculate 
vehicle speed, as well as pavement and suspension dynamics. These systems are commonly 
used to collect data for weight enforcement purposes (25). 


7.1.1.5 Magnetic Detectors 


Magnetic detectors indicate the presence of a metallic object through the detection of 
perturbation in Earth’s magnetic field. The installation of these detectors requires that either 
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pavement be cut or tunneling is used under the roadway, which in turn requires traffic 
interruption through a lane closure. They are typically unable to detect stopped vehicles (25). 


7.1.2 Nonintrusive Devices 


Nonintrusive devices are typically systems that require minimal disruption to traffic when 
installed and maintained. They are placed above ground, either overhead or on the side of the 
roadway. Types of these devices include microwave radar, laser radar, passive infrared, 
ultrasonic, and passive acoustic array (25, 52). 


7.1.2.1 Microwave Radar 

Microwave radar devices utilize a radar sensor mounted over the middle of a lane or at the 
side of a roadway. These devices transmit a continuous Doppler waveform that can provide 
measurements of vehicle count and speed (52). 


7.1.2.2 Laser Radar 

Laser radar devices utilize a sensor that is typically mounted over a lane, which can provide 
vehicle data on volume, speed, length of the vehicle, and classification. Modern laser radar 
sensors can produce two- and three-dimensional imagery of vehicles (52). 


7.1.2.3 Passive Infrared Sensors 

Passive infrared sensors detect energy through two sources: (1) energy emitted from vehicles, 
road surfaces, and other objects in their field of view and (2) energy emitted by the 
atmosphere and reflected by vehicles, road surfaces, or other objects into the sensor aperture. 
These sensors can be mounted overhead of traffic or on the side of the roadway. Multiple 
detection zones of the sensor are needed for it to measure vehicle speed and length (52). 


7.1.2.4 Ultrasonic Sensors 

Ultrasonic sensors transmit pressure waves to sound energy that are above the human audible 
range. These sensors can be placed adjacent to the roadway or mounted overhead. Ultrasonic 
sensors Can measure speed when programmed or placed in a correct manner to do so (52). 


7.1.2.5 Passive Acoustic Array Sensors 

Passive acoustic array sensors detect acoustic energy or audible sounds produced by vehicles 
and are able to count vehicles and measure their speed. These devices can be placed on the 
side of a roadway at specific angles and distances from the vehicles on the roadway to collect 
data accurately (52). 


7.1.3 Off-Roadway Devices 


Manual data collection equipment, such as radar guns, laser guns, and stopwatches, are still 
widely utilized for temporary data collection (51). In the Commonwealth of Massachusetts, 
these types of devices are most often utilized in the data collection process for the speed 
limit-setting process (1). Often, permanent data collection devices are not already at a 
location where the data needs to be collected, and installing a permanent device is 
unnecessary, given the need for short-term data. In this section, laser guns, radar guns, and 
stopwatch manual counts are described. 
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7.1.3.1 Laser Gun 

Laser guns, or LiDAR guns, use light detection and ranging technology that emit a series of 
infrared laser light pulses. When pointed toward a moving vehicle, this device is able to 
measure both the range and speed of the vehicle. This technology can be programmed to 
work in inclement weather and work through glass, though it does have a narrow field of 
view to report selected vehicles (51). When used according to correct guidelines, these 
devices are accurate within +1 mph to -2 mph (48). 


7.1.3.2 Radar Gun 

Radar guns can measure speed when pointed at a moving vehicle in the line-of-sight. These 
devices have a wide field of view and are programmed to report the fastest vehicle in its view 
(51). When used according to correct guidelines, these devices are accurate within +1 mph to 
-2 to -3 mph (49). 


7.1.3.3 Stopwatch Manual Count 

Manual counts, also known as running speed, are done using a stopwatch and two known 
start and stop locations. An observer starts and stops the stopwatch as a vehicle enters and 
departs the specified points marked in the section. The speed of the vehicle captured is then 
calculated dividing the distance between the two marked points by the time recorded (51). As 
humans have inconsistent and less accurate reaction times than automated devices, this type 
of count is rarely used for any type of engineering study. 
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7.2 Appendix B: Traditional Speed 
Collection Techniques 


Table 7.1: Ground Truth 


Timestamp | TL-TR | TL-BL_ TL-BR | TR-TL | TR-BL | TR-BR | BL-TL | BL-TR' BL-BR_ BR-TL | BR-TR_| BR-BL 


7:00:00 1 11 5 1 19 0 12 20 0 25 0 7 
7:09:20 0 5 3 0 10 0 17 16 1 14 0 1 
7:19:00 5 4 13 1 12 0 15 47 11 34 0 9 
7:28:20 2 3 8 2 10 0 25 28 3 26 1 9 
7:41:05 4 10 7 6 19 0 32 47 7 48 0 9 
7:50:26 4 9 5 1 17 1 33 26 1 20 2 7 
8:00:00 5 12 15 2 19 1 25 42 4 37 2 14 
8:09:20 4 2 3 6 14 1 12 32 3 23 0 6 
8:21:00 6 15 19 4 26 1 20 40 4 23 4 11 
8:30:20 7 10 9 5 18 1 18 29 6 23 4 4 
8:47:41 8 16 11 2 27 0 34 46 5 43 3 14 
8:57:14 2 3 7 0 5 0 7 9 1 19 1 3 


Table 7.2: True Positive 


Timestamp | TL-TR | TL-BL_ TL-BR | TR-TL | TR-BL | TR-BR | BL-TL | BL-TR'§ BL-BR_ BR-TL | BR-TR | BR-BL 


7:00:00 1 7 4 1 13 0 8 16 0 19 0 5 
7:09:20 0 4 3 0 9 0 13 12 1 12 0 1 
7:19:00 5 3 11 1 11 0 14 45 11 32 0 8 
7:28:20 2 3 8 2 9 0 24 25 3 23 1 8 
7:41:05 4 10 7 5 19 0 30 47 6 43 0) 8 
7:50:26 4 9 5 1 16 1 29 24 1 19 2 7 
8:00:00 4 11 13 2 17 1 23 38 4 37 2 12 
8:09:20 4 2 3 5 14 1 10 30 3 23 0 6 
8:21:00 6 14 18 4 23 1 17 36 4 20 3 10 
8:30:20 6 9 9 5 17 1 15 26 5 20 4 4 
8:47:41 8 16 10 2 25 0 33 41 4 38 3 12 
8:57:14 2 3 7 0 4 0 6 8 1 18 1 3 
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Table 7.3: False Positive 


Timestamp | TL-TR | TL-BL_ TL-BR | TR-TL | TR-BL | TR-BR | BL-TL 


BR-TR | BR-BL 


BL-TR  BL-BR | BR-TL 


0 
0 
1 
0 
0 
1 
0 
0 
1 
1 
0 
0 


7:00:00 
7:09:20 
7:19:00 
7:28:20 
7:41:05 
7:50:26 
8:00:00 
8:09:20 
8:21:00 
8:30:20 
8:47:41 
8:57:14 
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7.3 Appendix C: Quoted Cost Sheets 


Traffic analysis service ©> DATA FROM SKY 


Extraction of trajectories from aerial video data 


Based on the estimated traffic flow (min. trajectory 80 meters, max. 800meters) 


Georegistration (start fee) € 150,00 per task / intersection 
Video merging operation in time € 15,00 per operation 
Trajectories merging operation on request 

Low traffic fiow (up to 1500 vehicles/hour) € 150,00 per hour 

Medium traffic flow (up to 4500 vehicles/hour) € 300,00 per hour 

High traffic fiow (up to 8000 vehicles/hour) € 600,00 per hour 

Very high traffic flow (8000 vehicles/hour and more) € 800,00 per hour 


Extra: Pedestrians + bicycles trajectory extraction 


Low fiow € 50,00 per hour 
High fiow € 100,00 per hour 
Notes: 


The accuracy of speed and acceleration data depends on the quality of video and georegistration process. 
Trajectory merging operation in space is also possible. Contact us for detailed information. 

For demonstration purposes, a demonstration analysis of five minutes video is free of charge. 

Results of the analysis: TLGX file, which can be opened through SW tool DataFromSky Viewer 
DataFromSky Viewer is licensed software - 3 commercial types (Basic, PRO, Ultimate) 


DataFromSky is a product of RCE systems s.r.o. 
RCE systems s.r.0. | Svatopluka Cecha 1d | 612 00 Brno | Czech Republic | +420 775 345 465 | www.datafromsky.com 
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Innovative Data. UC 
P.O. Box 468 
Belchertown, MA 01007 


413.668.5094 


Innovative Data. UC is WRE/DRE certified in MA. 


Location: Amherst, MA - Double Roundabout at Atkins 

License Plate Capture 

90 Minute plate capture with 8 technicians 

License Plate Transcription into Excel 

Plate Matching Report 

Data to be delivered electronically only. Please allow up to two business weeks for 
electronic delivery of data. We will do our best to deliver the data as soon as 
possible, however, if data is requested to be delivered within 48 hours of data 
collection completion, there will be a 15% overtime expedition fee added to the 


counts. Hard Copy report to be mailed at no additional charge upon request. 
The total project cost includes all travel and data processing charges. 


If a police detail is required from the municipality for the work, the fee will be charged 


To reserve your scheduled count, Please email to 
ee com or fax to 413-485-7340. Thank you! 
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Executive Summary 


Unmanned Aerial Systems (UAS) —commonly referred to as “drones”— have emerged over 
the last two decades as a promising technology with a wide range of applications for military 
and police forces, emergency and public services, commercial enterprises, and recreational 
users. This research focuses on UAS applications in emergency response. It aims to provide a 
comprehensive review of existing work of using UAS for emergency response and to 
develop a UAS network to facilitate such UAS applications. This report documents the 
research efforts, and consists of two main components: the first one is the literature and the 
second one presents a procedure to develop a UAS network for emergency response in 
Massachusetts. This procedure is very flexible and can be adapted for applications in other 
states. 


This report begins with providing the background of the research task and objectives in 
Chapter 1. Chapter 2 summarizes the findings of the literature review, which focuses on the 
public policy and administration aspects of UAS applications in the United States, 
particularly in Massachusetts. The Chapter also looks at recent UAS applications for 
emergency response both nationally and internationally, key issues related to the planning of 
UAS emergency response networks, as well as the current UAS practices regarding 
emergency response of both MassDOT and the Massachusetts Emergency Management 
Agency (MEMA). The literature review suggests that current federal policies encourage the 
exploration of using UAS for different public purposes, and there has been growing interest 
in applying UAS for emergency response, including traffic incident management and 
emergency preparedness and response. Based on the literature review, an assessment of 
MassDOT needs for UAS in emergency response is performed and the results are also 
included in Chapter 3. 


Chapter 3 presents the procedure to develop a UAS network for emergency response. It 
begins with introducing the geospatial datasets used, including historical traffic incident and 
natural disaster data (e.g., hurricanes and flood zones). Traffic incidents data from 2013 to 
2017 is kindly provided by MassDOT and is used as one of the inputs for the proposed 
analysis procedure. Based on the data, a two-step method is proposed to minimize the cost 
and maximize the coverage of a UAS network for emergency response. The first step is to 
identify incidents that are well-suited for using UAS and the second one develops an 
optimization algorithm to determine UAS network parameters, including the number of UAS 
stations needed and their locations. This research further applies the developed algorithm to 
the applicable incidents identified, to find optimal UAS network parameters given some 
assumptions regarding the coverage area of each UAS station (i.e., station coverage radius). 
Not surprisingly, it is found that the UAS network parameters vary with the station coverage 
radius assumption and the target coverage rate. Specifically, the number of UAS stations 
needed increases with the target coverage rate but decreases with the station radius. The 
proposed procedure is designed to be generic and flexible. For practical UAS network 
decision-making with different traffic incident types and a heterogeneous fleet of UAS, the 
procedure can still be applicable with minor modifications. 
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Finally, Chapter 4 summarizes the key findings from this research task and briefly discusses 
future research needs. 


In short, this report is an in-depth look at and analysis of Massachusetts “highway” incidents, 
particularly as they relate to the potential for drone use. This analysis was an important step 
in the MassDOT Drone Program’s incident response development and an aid to pre-planning 
for drone services of this type. By analyzing the incident types, severities, frequencies, and 
locations of highway incidents over the period April 2013 to October 2017, the research team 
has defined the concept of “key stations” that would “see” the largest number of incidents for 
a given drone launch point. 26 of these would be able to cover 95% of incidents statewide, 
and 13 of these would cover 81%. The key takeaways of this research are: 
e “Key station” concept definition, station locations, and utility analysis; 
e Decomposition of highway incidents over 4+ year period to serve as a reference for 
future drone utility assessments; and 
e Overlay of “other disaster events” to investigate the coupling of environmental 
disasters (hurricanes, flooding) as they affect drone response capability and 
Massachusetts transportation infrastructure. 


For the overall value of this research, while only an initial step in developing a robust 
statewide drone incident response capability, it provided key products and conclusions that 
will allow the drone team to build on the takeaways mentioned above. Since the delivery of 
the draft report, the research has been referenced and cited several times in support of 
subsequent drone team projects. 
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1.0 Introduction 


Unmanned aircraft systems (UAS) offer many potential opportunities to assist with surface 
transportation needs within the Massachusetts Department of Transportation (MassDOT), as 
well as at other state agencies. One promising application is to use UAS for emergency 
services and disaster relief. MassDOT and the Massachusetts Emergency Management 
Agency (MEMA) are both looking for information that will help them develop a better 
understanding of UAS emergency management support capabilities, including the ability to 
obtain rapid and critical post-disaster information to support both lifesaving and damage 
assessment services. 


1.1 Objectives 


The main objective of this research task is to conduct a literature search and detailed 
synthesis of the application of UAS in emergency response and disaster damage assessment 
services on surface transportation networks. 


The research task (Task D of the MassDOT-funded UAS study) had two deliverables: 
e Deliverable D1: Technical Memo on Literature Review 
e Deliverable D2: 
a) A GIS-based hotspot map of previous incidents and natural disasters in 
Massachusetts. 
b) An initial UAS deployment network for emergency response. 
c) Python code used to compile data and create the initial UAS deployment network. 
d) A report describing the methods to identify applicable events for UAS missions, 
development of the UAS network, and the potential ways to use the UAS network 
for decision-making. 


As Task D progressed, the research team received feedback from MassDOT and MEMA. 
Based on such feedback, the team deviated slightly from the original Scope of Work (Tasks 
D2 and D3) and revised Deliverable D2 to be more practical and implementation-focused. 


This page left blank intentionally. 


2.0 Literature Review and Assessment of MassDOT 
Needs 


In this section, we first review relevant literature in three areas: policy and administration 
aspects of UAS applications (Part 2.1); existing applications of UAS for emergency response 
(Part 2.2); and UAS for emergency preparedness (Part 2.3). We also report our findings from 
a mid-stage project discussion with MassDOT officials and the questions they have regarding 
implementing UAS for emergency response in Massachusetts (Part 2.4). We close the section 
with some reflections from the review. 


2.1 Policy and Administration Aspects of 
UAS Applications 


Unmanned Aerial Systems (UAS) — commonly referred to as “drones” — have emerged 
over the last two decades as a technology with applications for military and police forces, 
emergency and public services, commercial enterprises, and recreational users. For any 
report providing advice on the possible use for emergency services, it is important to first 
provide an overview of the current legal, policy, and administrative environments 
surrounding UAS technologies with a particular lens toward use in emergency situations. 


2.1.1 Summary of Current Federal UAS Policies 


Increased demand for airspace, driven by the rapid emergence of UAS technology, pushed 
the United States Congress, in 2012, to pass the Federal Aviation Administration’s (FAA) 
Modernization and Reform Act of 2012 (Public Law PL 112-95). This Act requires the 
development of published rules for drone operation in domestic airspace by 2015 (1). The 
FAA regulations that emerged, allow the recreational small drone market to continue to 
proliferate and require commercial and public entities to seek FAA authorization before 
testing and utilizing drones for various applications. 


There are several methods in place for private, commercial, and state or local entities to gain 
permission to operate UAS. The FAA publishes guidance online for model aircraft and small 
UAS weighing up to 55 pounds. Agencies in Massachusetts interested in operating UAS 
should be familiar with Part 107 Small UAS Rules, Certificates of Waiver or Authorization 
(COA), and Special Government Interest (SGI) expedition waivers (2). 


2.1.1.1 UAS 

Drones weighing 0.55 pounds or less typically reach a maximum height of 100 feet and 
cannot operate beyond visual line of sight (BVLOS) of the operator and wireless radio 
controller. These drones are frequently flown by hobbyists and other amateurs, including 
children. Due to the weight and size limit, it is unlikely that these drones will interfere with 
air traffic in surrounding air space and airports. These operators do not need to be certified 


UAS pilots but should still know the rules published for hobbyists with drones as heavy as 55 
pounds (3). 


2.1.1.2 UAS Requiring a Certified Pilot; Small UAS Rule (14 CFR 107) - Part 107 
Emergency and public service organizations throughout the country can operate drones under 
the Part 107 rules (4), or seek authorizations and waivers supporting regular and emergency 
UAS operations. In general, the FAA notes that “most of the restrictions... are waivable if 
the applicant demonstrates that ... operation can safely be conducted under the terms of a 
certificate of waiver.” 


To operate under Part 107, the operator must attain a remote pilot certificate from the FAA, 
register the UAS as a ‘non-modeler,’ and follow the Part 107 rules briefly summarized below 
and in Table 2.1. 


“To obtain a remote pilot certificate, an individual must be 16 years of age or older; be able 
to read, speak, write, and understand English; be in physical and mental condition to operate 
a small UAS; and pass an aeronautical knowledge exam at an FAA-approved knowledge 
testing center. The approved license must be accessible by the pilot and during all UAS 
operations and is valid for two years.” (FAA Part 107) 


Table 2.1: Summary of FAA Part 107 flight operation rules 


Rule # FAA Part 107 Flight Operation Rules 


1. Fly in Class G airspace (otherwise unrestricted airspace below 1,200 feet) 


Keep the unmanned aircraft within visual line-of-sight 
Fly at or below 400 feet 


2 

3 

4. Fly during daylight or civil twilight 
5. Fly at or under 100 mph 
6 

z, 

8 


Yield right of way to manned aircraft 


Do not fly directly over people (without permission) 


Do not fly from a moving vehicle, unless in a sparsely populated area 


2.1.1.3 Certificate of Authorization 

State and local officials can apply to the FAA for a Certificate of Waiver or Authorization 
(COA) (5). The flight rules are identical to those in Part 107. Registering for a public COA 
includes reporting the type of UAS program and training completed to the FAA. 


2.1.1.4 Special Government Interest (SGI) 

The FAA may expedite approval for emergency drone operations under a Special 
Government Interest (SGI) process (6). To be eligible for an SGI waiver, an individual must 
be an existing Part 107 Remote Pilot with a current certificate or must have an existing 
Certificate of Waiver or Authorization (COA). If approved, an amendment will be added to 
the existing COA or Remote Pilot Certificate, which authorizes the pilot to fly under certain 
conditions. SGI waiver-seekers need to complete an Emergency Operations Request Form 
(Figure 2.1), and in return will receive the cooperation and coordination assistance of the 
FAA. 


¢ ay FAA 
Ait Irate. Organaeton 
~ A: J Syuem Opermors Securty 


FAA REQUEST FORM FOR 
EXPEDITED SGI WAIVER OR AUTHORIZATION FOR UAS OPERATION 


Basic Qualifications 


@ The requesting operator must possess a Certificate of Waiver or Authorization (COA) or Part 107 Pilot License 

@ The UAS operation must support an emergency response or other effort being conducted to address exigent 
circumstances and that will benefit the public good 

@ The requested FAA approval cannot be secured via normal processes in time to meet urgent operational needs 


Requester/Operator Information 


Click here to enter text. 


Click here to enter text. 
Click here to enter text. 


Documentation 


Click here to enter text. 


Nature and Description of Event 


Firefighting / Law Enf t/ Click here to enter text. 
Search and Rescue 


SOSC 2018/02/08-15002 


Local / National / Natural Disaster 
Critical Infrastructure 


Requested Flight Details 


Click here to enter text. 
Click here to enter text. 
Click here to enter text. 


Click here to enter text. 


For those flights remaining within a general contiguous area, 
which cannot be easily described as a circular shape, provide 


Click here to enter text. 


Additional Pilot Ratings 


Click here to enter text. 


SOSC 2018/02/08-15002 


Source: U.S. Federal Aviation Administration, Certificates of Waiver or Authorization (COA), March 9, 2018. (5) 


Figure 2.1: FAA special government interest (SGI) waiver request form 


2.1.1.5 Section 333 Exemptions 

Commercial entity applications such as real estate appraisals, bridge inspections, and movie 
cinematography can all apply for permission to use the national air space. These 
authorizations are granted under Section 333 Exemptions (7). These commercial use 
exemptions applied to small UAS and are granted on a case-by-case basis. Typically, the 
FAA approval process for a Section 333 Exemption takes up to 90 days. 


2.1.2 Summary of Massachusetts UAS Policies 


No statewide law has been added to the federal rules. However, some towns and other 
governmental entities have implemented additional UAS restrictions, as summarized in Table 
2.2. Some municipalities, such as the Towns of Chicopee and Holyoke, passed ordinances to 
restrict UAS usage in their jurisdictions, but these are likely an overreach as the FAA has 
preemptive authority on the national air space. In an earlier court case, a federal judge in 
Massachusetts overtumed key parts of a Town of Newton ordinance, which restricted where 
drones could fly; the judge stated that the town would need to re-draft the ordinance to 
adhere to federal law (8). 


Table 2.2: Summary of UAS codes in Massachusetts 


Town of Barnstable (9) Banned UAS on beaches, following the lead of the Cape Cod 
National Seashore 


Quabbin Reservoir (MA 
Dept of Conservation and 
Recreation) (10) 


Restricts UAS in the Quabbin Reservoir without prior written 
approval 


Cape Cod National 
Seashore (U.S. National Bans UAS at Cape Cod National Seashore beaches 
Park Service) (11) 


UAS cannot fly or take off/land on private property without prior 


Town of Chicopee (12) written consent; reinforces 5 mile radius to local airfield 


UAS cannot fly or take off/land on private property without prior 
written consent 


Town of Holyoke (13) 


2.1.3 Discussion: Key Policy and Administration Topics 


Beyond the legal framework for operating UAS in Massachusetts, there are important ethical 
and administrative issues on how to best implement these systems as discussed below. 


2.1.3.1 Privacy 

Privacy has an evolving definition. In public service, UAS applications may inadvertently 
capture private information (14) (e.g., facial images, vehicle license plates) in different 
situations, such as field surveys and law enforcement use. To date, there is little relevant 
guidance on managing the data collected, including how to store the data, how long to keep 


the data, and who should have access to the data (15). State and local officials should explore 
and develop policies for managing UAS collected data in an appropriate way. 


2.1.3.2 Federalism 

Many Americans support state and local initiatives to exercise authority over UAS use in 
their communities (16), including organizations such as the United States Conference of 
Mayors, and the National Governors Association (17). The National Conference of State 
Legislatures tracks UAS policies closely (18). The Drone Federalism Act of 2017 introduced 
in the 115th Congress in May 2017 sought to increase local control over the skies for VAS 
operations (19). Many of the concerns focus on recreational and commercial UAS, not 
emergency applications. While the United States Congress and the FAA support states using 
UAS for many applications, the development of UAS regulations at the local levels could 
introduce future change for potential UAS applications. 


2.1.3.3 Safety 

More UAS in the skies increase the likelihood of collisions both on the ground and in the air 
with vehicles, infrastructure, and people. Unmanned aircraft traffic management is “regarded 
as a key component of safe UAS integration into the national airspace system” (20). 
Currently, the FAA restricts UAS flights above people (without consent) or automobile 
traffic. Research also supports this restriction as it was found that UAS can contribute to 
driver distraction (21). Public safety will remain a constant concern in UAS applications. To 
enhance safety, one potential option could be to set a high standard on UAS pilot certificates. 
The FAA maintains a legal limit threshold for UAS operators, which includes an age 
requirement and an online test. Currently, someone without any UAS experience can be 
certified to operate drones. This is a low standard compared to US Air Force UAS operators 
who are all trained pilots (22). Massachusetts should consider mandating a higher level of 
training for its UAS pilots, especially those who may pilot a drone during an emergency. 


2.1.3.4 Spectrum Allocation 

All drones for commercial usage are controlled with communication systems utilizing radio 
frequency (RF) spectrum, which are currently allocated for AM/FM transmissions and used 
by the United States Department of Defense (23). There are ongoing discussions about 
reclassifying portions of the RF spectrum for emergency service use. If reallocation occurs 
outside the current frequency range, states will need to accommodate this change. 


2.1.4 Recommendations 


The FAA has the support of the United States Congress and the American people, suggesting 
that UAS technologies will become integrated into the national airspace in time. In this 
supportive environment, state policymakers are open to exploring the many applications of 
drones. The current guidelines are likely to remain valid at least in the short-term. Therefore, 
Massachusetts should consider taking steps to comply with, and master the FAA guidance of 
Part 107 (the Small UAS Rule) as small crafts are the most common drones available. 
MassDOT should also establish a system to allow the rapid filing of Special Government 
Interest flight waivers. Additionally, we recommend that Massachusetts create a pilot 
training and testing program beyond the standard requirement of an FAA 107 test to train a 
cohort of skilled and knowledgeable UAS pilots that can safely operate drones for emergency 


service. We also recommend that Massachusetts policymakers continually address 
cybersecurity issues related to UAS. The Commonwealth of Massachusetts should consider 
establishing a statewide data management system for UAS data to protect privacy. Such a 
management system should explicitly define what data can be used and how, who has access 
to the data, how to securely store the data, and when to delete it. This statement relates to a 
separate research task, Task F, of the larger MassDOT UAS study, examining potential 
applications of UAS in surface transportation (please see Volume II-F for the Task F report). 


2.2 Existing Applications of UAS in 
Emergency Responses 


Drones are increasingly being used for emergency response related to natural disasters. In 
addition, many agencies are interested in exploring the potential of UAS for traffic incidents. 
This section contains the research team’s review of existing UAS applications in these two 
areas. 


2.2.1 UAS Applications for Natural Disasters 


Table 2.3: Summary of drone application in natural disasters 2005-2017 


Application 
Year Natural disaster Country | Name of drone 
A B 
Hurricane Katrina 
2005 | Response(USA) UAS AeroViroment Raven v v 
Evolution v Vv 
iSENSYS T-Rex v ¥ 
Silver Fox v ¥ 
Hurricane Katrina 
2005 | Recovery(USA) UAS iSENSYS IP3 
Hurricane Wilma 
2005 | (USA) UAS iSENSYS T-Rex v 
Berkman Plaza 
2007 | II(USA) UAS iSENSYS IP3 
Laquila Earthquake 
2009 | (Italy) Custom Vv 
Haiti Earthquake 
2010 | (Haiti) Elbit Skylark ¥ 
Christchurch 
2011 | Earthquake(NZ) Parrot AR. Drone 
Tohoku Earthquake 
2011 | (Japan) Pelican 
Fukushima Nuclear 
2011 | Emergency (Japan) Honeywell T-Hawk v 
Evangelos Florakis 
2011 | Explosion (Cyprus) AscTec Falcon Vv 
AscTec Hummingbird Vv 
Thailand Floods 
2011 | (Thailand) FIBO UAV-1 v 
FIBO UAV Glid v 
Finale Emilia 
2012 | Earthquake(Italy) NIFTi 
Typhoon 
2013 | Haiyan((Philippines) Unknown 
Lushan Earthquake HW18 (Ewatt 
2013 | (China) HoverWings) Vv Vv 
Yuyao 
2013 | Flooding(China) River-map UAV Vv 
Boulder Colorado 
2013 | floods(USA) UAS Falcon Fixed v 
SR350 Mudslides 
2014 | Response(USA) UAS DJI Phantom v 
AirRobot 100 v 
Precision Hawk ¥ 
SR350 Mudslides 
2014 | Recovery(USA) UAS AirRobot 180 


Precision Hawk v Vv 
Balkans flooding 
(Serbia, Bosnia- 
2014 | Herzegovina) ICARUS custom v v 
Collbran landside 
2014 | (USA) UAS Falcon Fixed Vv Vv 
Falcon Hover Vv Vv 
Yunnan China 
2014 | Earthquake(China) Parrot AR Type 2 v 
Dar Es Salaam 
2015 | flood(Tanzania) Ebee v 
Nepal 
2015 | Earthquake(Nepal) Unknown v v 
Islands of Vanuatu 
2015 | cyclone Pam Indago quadcopters Vv 
Alliance hexacopters Vv 
fixed wing Trimble v 
2016 | Ecuador earthquake Unknown v v 
Southern California 
“Skirball” Wildfires 
2017 | (USA) UAS DJI Matrice 100 Vv Vv 
2017 | Mexico Earthquakes Unknown Vv Vv 
Hurricane Flying Cow(or Cell on 
2017 | Maria(USA) UAS Wings) v 
Hurricane 
2017 | Irma(USA) UAS DJI Phantom 3 ¥ 
Parrot Disco v v Vv 
Hurricane 
2017 | Harvey(USA) UAS Unknown Vv Vv Vv 
Flooding and 
Mudslide in Sierra 
2017 | Leone Unknown Vv Vv 
Mudslide in 
2017 | Mocoa(Colombia) Aeryon SkyRanger Vv Vv 


Notes: A=Search, B=reconnaissance and mapping, C=structural inspection, D=estimation of debris, 
E=Communication 


The team examined recent (2005-2017) applications of UAS in seven types of natural 
disasters: fire, earthquake, flood, hurricane, volcanic eruption, landslide, and search and 
rescue operations. In those emergency situations, drones offered unique advantages in five 
dimensions, including search, reconnaissance and mapping, structural inspection, estimation 
of debris, and communication. Table 2.3 provides a summary of drone applications in natural 
disasters in recent years. Different types of drones, fixed wing and rotary, in different shapes, 
sizes, and capacities, were used in those events. The takeaway from this part of the review is 
that the characteristics and thus the capabilities of drones vary significantly and that agencies 
should select drones based on the desired applications. 
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2.2.2 UAS Applications Specifically for Traffic Incidents or Emergencies 


According to a 2012 report on energy security and traffic congestion, bottlenecks (40%) and 
traffic incidents (25%) are the two most significant causes of traffic congestion in the United 
States (U.S.) (24). Transportation agencies and first responders are always looking for 
innovative and cost-effective ways for resolving traffic incidents. 


Stevens (25) has reviewed the applications of UAS for traffic incident management (UAS- 
TIM) across different state Department of Transportation (DOT) agencies in the U.S. and 
applications in other countries. It was found that over ten state DOTs looked at, or were 
looking at UAS-TIM. Among them, a few canceled the projects due to constraints from FAA 
regulation, while several (such as in Virginia, Ohio, Wyoming, and Utah) conducted test 
flights to ascertain the feasibility of UAS for transportation applications. None of the state 
DOTs have made UAS-TIM a routine practice. Outside the U.S., Norway and China tested 
applications of UAS for crash mapping, but the projects did not result in implementation. Lee 
et al. (26) reviewed research that used full-size fixed-wing UAS for traffic surveillance and 
found that most studies had not yet been implemented in the field due to safety concerns and 
regulatory issues. 


While field tests of UAS for traffic management are limited, two applications are worth 
noting. In a joint project with the Metropolitan Transit Authority of Harris County and 
Houston TranStar, Stevens and Blackstock (27) conducted a demonstration project to 
examine UAS-TIM. Through several live demonstrations on highways, they showed that 
UAS can meet three needs (incident monitoring, situational awareness, and quick clearance 
and recovery) in traffic accident management. The two UAS platforms used in their study 
were: (1) a DJI Inspire 2 (untethered) UAS with DJI ZENMUSE XS5S gimbal and camera 
system; and (2) a CyPhy Works Persistent Aerial Reconnaissance and Communications 
(PARC) tethered UAS with Long-Range Zoom Electro-Optical/Infrared (EO/IR). This study 
confirmed that UAS can perform all of the following: 

e Real-time confirmation of traffic incidents 
Real-time monitoring of traffic incidents 
Real-time monitoring of alternate routes 
Real-time monitoring of traffic incident queuing 
Real-time monitoring of secondary crashes 


Stevens and Blackstock (27) expected that their UAS will fit into the traffic incident 
management system as “other traffic management” to provide and receive data about road 
network conditions, traffic images, and incident information. However, the report (27) also 
pointed out that the demonstration did not result in regular use of UAS for TIM in the partner 
agencies. This same report (27) also raised questions regarding UAS’s crash scene mapping 
capability and whether the quality of UAS images can meet the requirements of court 
proceedings. 


Lee et al. (26) conducted a pilot study that used quadcopter drones for incident monitoring. 
In their framework, after a traffic incident occurs, the highway patrol team arrives at the 
scene and quickly deploys drones with first person view (FPV) cameras to capture the 
incident scene. The drones send the videos to the ground station through a 2.4 GHz radio 
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communications link. Through a commercial 4G/LTE network, the videos are then 
transmitted to the remote Traffic Control Center (TMC). They found that the drones were 
beneficial in that they could cover a wide area, which enabled queue length and delay 
measurement, and that they could do instant incident monitoring. Lee et al. (26) identified 
several challenges related to the applications of UAS for traffic surveillance and incident 
monitoring. One was the limited drone flying time. They recommended more expensive 
quadcopters that can stay in the air for more than an hour with up to 55 pounds of payload. 
Another one was limited video quality: it was fine for manual data processing but not 
sufficient for video analytics. Lee et al.’s work also found that full-size UAS have not often 
been implemented for traffic surveillance in the field much due to safety concerns and 
regulatory issues. 


Overall, it is well recognized that drones have great potential for traffic incident 
management, including for providing situational awareness, monitoring traffic, and collecting 
data. At the same time, there are challenges associated with these applications, including the 
quality of UAS images, payload capacity, flight endurance, and tolerance to wind and 
turbulence (27, 28). 


2.3 UAS for Emergency Preparedness 


In this section, the research team reviews the planning aspects of UAS and UAS network 
design for emergency preparedness and response. We first provide an overview of federal 
disaster planning directives, a review of the literature available on UAS for disaster planning, 
and our recommendations to MassDOT on using UAS for both disaster and long-range 
transportation planning. After that, we review the literature on UAS network design, a critical 
component needed to enable UAS use, which can help plan and design a UAS network in 
Massachusetts for emergency preparedness. 


2.3.1 Planning for Emergencies 


Disaster planning in the United States is predicated at the federal level through Presidential 
Policy Directive 8 (PPD-8) (29), the Disaster Mitigation Act of 2000, and the Pre-Disaster 
Hazard Mitigation Act of 2010 (30). PPD-8 calls for the creation of a National Preparedness 
Goal (NPG) that supports preparedness efforts through guidance from the federal 
government to local governments via pre-disaster recovery planning (31). The intent of both 
the above statutes is essentially to provide a structure for the administration of disaster relief 
while controlling costs to the federal government from these efforts. States and local 
governments are able to seek federal funding for disaster relief and recovery upon meeting 
specific federal eligibility requirements, with development of a Federal Emergency 
Management Agency (FEMA)-approved hazard mitigation plan of the primary obligations. 


For the federal government, effective pre-disaster planning is a process that integrates federal 
and local community planning objectives with the goal of facilitating decision-makers with 
the ability to reach sound decisions and investments. There are six (6) planning steps and 
nine (9) key recovery activities (Figure 2.2) that serve as guidance for pre-disaster planning 
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activities (31). The six planning steps include (i) form a collaborative planning team, (ii) 
understand the situation, (iii) determine goals and objectives, (iv) develop the plan, (v) 
prepare, review, and approve the plan, and (vi) implement and maintain the plan. 


STEPS KEY ACTIVITIES _ 


Forma Define collaborative planning team and scope of planning activities 
Collaborative 
Planning Team 


Develop and implement partner engagement strategy 


Understand 7 
the Determine community risks, impacts, and consequences 


Situation 


Determine 
Goals and Assess community's capacity and identify capability targets 


Objectives 


Develop Determine leadership positions and define operations necessary 


the Pl 
x ances Establish processes for post-disaster decision-making and policy setting 


Prepare, Review, Write the local pre-disaster recovery plan 
and Approve 
the Plan 


Approve the pre-disaster recovery plan and associated regulations 


Implement 
and Maintain identify ongoing preparedness activities 


the Plan 


Source: Federal Emergency Management Agency, Pre-Disaster Recovery Planning Guide for Local 
Governments, FD 008-03, 2017 (31, p. 27). 


Figure 2.2: FEMA pre-disaster recovery guide for local governments 


The plan-making process for preparing communities and organizations to adequately manage 
an emergency include operational plans (limited pre-disaster), policy plans (pre-disaster) and 
recovery plans (post-disaster). (32) 


2.3.1.1 UAS for Disaster Planning 

A critical component of all planning is the ability to acquire and analyze data. Planners have 
an extensive toolbox to draw from for various types of information that can be used to 
develop and test scenarios that are then presented to stakeholders for critique and refinement. 
Monitoring and appraisal of how plans are implemented is a tedious endeavor that only 
becomes more difficult over time, and a significantly cumbersome process when there is little 
or no current data available. 


During disaster events, multiple parties have to make quick decisions to protect life and 
property. UAS can provide a low cost and quick turnaround option, that provides high- 
quality visual documentation of conditions as events unfold. UAS can be an important tool 
for disaster management for first responders, planners, and elected officials. 


A uniform planned standard for the use of UAS across disciplines is not currently available. 
To date, no one has developed a comprehensive framework to explore the full extent of 
potential UAS applications as part of the pre-disaster planning process. As a result, present 
use of UAS for such planning is limited. 
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However, the potential of UAS for disaster applications is worth noting. Drones can provide 
transportation decision-makers with needed aerial support in areas and situations where it is 
too dangerous to deploy human assets to collect time-sensitive information. Kim and 
Davidson (33), conducted an analysis of how UAS can support critical transportation needs 
in disaster response scenarios by providing real-time video and photographic imagery of 
roads blocked by debris from downed trees, collapsed buildings, power lines, and so forth, to 
help determine which roads are passable for emergency response teams and which are 
possible evacuation routes. 


2.3.1.2 Disaster Planning for MassDOT 

Life shows that those with backup plans tend to recover and adapt from hardship at a faster 
rate than those that have not made hazard planning a priority. One of the lessons learned 
from Hurricane Maria in Puerto Rico in 2017 was that, when the hurricane made landfall, 
local and regional government agencies were all alone until the deployment of the Federal 
Emergency Management Agency and the Commonwealth of Puerto Rico rescue and 
recovery resources that took place only after weather conditions returned to normal. 


For MassDOT, there could be significant problems if a hurricane like Maria ever made 
landfall in Massachusetts. The disruptions caused by recent snowstorm events (Nemo in 
2013, and Juno in 2015) affected not only Massachusetts but the entire New England region. 
It is unrealistic to expect that there will be a plan for every single situation. However, without 
a contingency strategy in place, minor problems can escalate and affect multiple systems and 
networks beyond those under MassDOT’s control. 


The research team has the following recommendation to MassDOT: Incorporate the use of 
UAS as a planning and analysis tool for disaster management, with MassDOT’s Aeronautics 
Division acting as the main entity within MassDOT to administer and deploy UAS resources. 
UAS can be used to identify crucial pressure points in the critical infrastructure in a relatively 
short period of time prior to and after extreme weather events. Such valuable information 
could make a dramatic difference for returning life to normal for millions of people. UAS 
will be additionally valuable if, and when, they become a part of a comprehensive disaster 
planning strategy. 


2.3.2 UAS Network Design for Emergency Preparedness 


In the application of UAS for emergency situations, one critical factor is the UAS network 
design. Often multiple emergent events happen simultaneously or in a very short time period, 
which then requires a resilient UAS network that can handle multiple events in an extensive 
area. 


Erdelj et al. (34) identified several networking-related research challenges when using UAS 
for disaster management: (1) creating and maintaining the information relay network; (2) in- 
network data fusion; (3) handover issues; (4) UAS physical constraints compromising the 
communication; (5) automated network maintenance and UAS charging; (6) UAS network 
security and robustness; (7) UAS failure handling; and (8) privacy and trust issues. 

To the best knowledge of the research team, existing investigations on the use of UAS 
networks for emergency preparedness has only been conducted in academia and there is no 
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example yet put into practice. Nevertheless, a review of the academic findings may shed light 
on optimal UAS network design if MassDOT is to implement this UAS application. 


2.3.2.1 UAS network architecture design 

Morgenthaler et al. (35) introduced UAVNet, an architecture and prototype implementation 
of an autonomously deployable temporary and flying Institute of Electrical and Electronics 
Engineers (IEEE) 802.11s Wireless Mesh Network. The central communication components 
are the wireless mesh nodes carried by UAS. The devices on the ground are divided into two 
groups: the end systems intending to communicate with each other and the monitoring and 
configuration devices, such as iPhones or iPads. 


Erdelj et al. (84) presented a new perspective for classifying disasters and introduced a 
theoretical framework of wireless sensor-actor network architectures that can be effective in 
each of these cases for disaster management. In the example scenario, multiple deployed 
sensors, including drones and others, collect physical information (here, the water level at the 
monitored bank and vibration/displacement on the mountainside) and forward this for 
recording and storage at a centralized server location. The Wireless Sensor Network 
integrates different sensors with the monitoring displacements of landslides and triggers the 
alarm in the case of debris flow. The stand-by UAS can be called into active operational 
service. Schwab (32) designed another scenario in which the UAS form an independent 
network without support from the ground sensors. Multiple UAS stations that are 
strategically deployed over a wide geographical area can provide certain guarantees that 
some parts of the UAS infrastructure would operate even after a disaster occurs. 


2.3.2.2 Optimization of UAS Network Operation 

Bupe et al. (36) propose a fully autonomous system to deploy UAS as the first phase disaster 
recovery communication network for wide-area relief. An automation algorithm was 
developed to control the deployment and positioning of UAS based on a traditional cell 
network structure utilizing seven-cell clusters in a hexagonal pattern using MAVLink. The 
distributed execution of the algorithm is based on centralized management of UAS cells 
through assigning higher ranked UAS referred to as supernodes. The algorithm 
autonomously elects supernodes based on weighted variables and dynamically handles any 
changes in the total number of UAS in the system. This system represents a novel approach 
for handling a large-scale autonomous deployment of a UAS communications network. 
Bupe’s proposed autonomous communication network was verified and validated using 
software simulations and physical demonstrations using identical quadrotor UAS. 


Chowdhury et al. (37) proposed a Continuous Approximation (CA) model to determine the 
optimal locations for the distribution centers with their corresponding emergency supply 
inventories and service regions. The authors approximated drone transportation costs by 
considering a number of specific routing factors such as climbing, hovering, descending, 
turning, acceleration and deceleration, rotation, and constant speed cost. 


Cong et al. (38) modeled a UAS network optimal path problem by finding the least cost tour 


on a specified set of arcs in a graph, which is related to the Chinese postman problem and the 
rural postman problem. By mapping the real traffic network into a virtual network, they were 
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able to solve the problem by using mixed-integer linear programming. Their objective 
function aimed to minimize UAS battery usage, response time, and fixed UAS activation 
costs subjective to certain assignment constraints, such as different flying routes and UAS 
recharging depots. 


2.3.3 Summary 


UAS have great potential for emergency preparedness applications. However, a uniform 
planned standard for the use of UAS across disciplines is currently not available in 
Massachusetts. Therefore, the research team recommends that Massachusetts incorporate the 
use of UAS as a planning and analysis tool for both disaster and long-range transportation 
planning. 


Additionally, to fully utilize the potentials of UAS for emergency preparedness and response, 
UAS network design is a critical issue. We found that there are growing research efforts in 
academia that investigated both the architectural design of the UAS network and ways to 
optimize the operation of UAS networks (such as to minimize cost). However, the UAS 
networks in the reviewed studies were often simplified in order to provide a theoretical 
framework. Therefore, it is challenging to directly implement those networks in practice. 


2.4 Practice and Potential Needs of UAS in 
Massachusetts 


In order to connect the literature review described above to the potential needs regarding 
UAS in Massachusetts, the research team attended a meeting at MEMA in Framingham in 
July 2018. Attendees included staff from MEMA and the MassDOT Aeronautics and 
Highway Divisions. At this meeting, among other things, the researchers learned that 
MassDOT is actively making progress toward the use of UAS for emergency response. Also, 
MEMA reportedly uses drones, through contractors, to assess disaster damage. 


Building on the literature review findings and looking forward, it is clear that MassDOT 
could readily use UAS to respond to and/or assist investigations of traffic incidents, such as 
highway crashes and transit bus failures. For MEMA, important functions are to monitor 
infrastructure (such as coastlines or areas that are susceptible to flooding) before and after 
disasters to assess the damage, and monitoring transportation systems during an evacuation 
event, sometimes over a large geographic area. UAS can assist with these tasks. 


Further, the discussions in the July meeting helped the research team to identify and clarify 
important questions and needs of the MassDOT and MEMA officials if UAS are to be used 
for emergency response. The key questions raised in the meeting include: 


1. What are the parameters for a good drone mission? When — in what situations — 


should drones be used? And specifically, from a policy (privacy), FAA, and network 
perspective, where are the “good” drone missions? 
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2. Where, geographically in Massachusetts, should a fleet of UAS be located for rapid 
response to emergency incidents? 

3. Who should own the emergency response drones? MassDOT? Or should a network of 
contractors be inventoried and utilized? 

4. Who already has the UAS equipment and piloting capabilities to be called on in an 
emergency response situation? Currently, neither MassDOT nor MEMA has a full 
inventory of UAS licensed pilots and capabilities. And what additional training and 
testing systems assessing pilot skills are needed? 


The remaining of the research project will address some of the questions raised above, 
detailed in the following sections. 


2.5 Conclusions 


In this synthesis effort, the research team examined the public policy and administration 
aspects of UAS applications, the recent applications of UAS for emergency and incident 
response, issues around planning and network design of UAS for emergencies or disasters, 
and the practice and needs of UAS for emergency and incident response at MassDOT and 
MEMA. 

Our review suggests that current federal policies encourage exploration of UAS applications 
for different public purposes, including for emergency and incident response. The research 
team thus recommends that Massachusetts work on the policy and administration issues that 
are limiting the use of UAS for these purposes. Specific recommendations include 
establishing a UAS data management system for privacy and developing a pilot training 
and testing program that goes beyond the standard FAA 107 testing to establish and 
maintain a cohort of skilled UAS pilots who can safely and effectively respond in emergency 
situations. 


We also find that nationally there are growing interests in using UAS for emergency and 
incident response. UAS use is increasingly used during or after natural disasters, particularly 
in instances of fire and flooding. One important lesson learned is that the characteristics 
and capabilities of drones vary significantly, and agencies should select UAS equipment 
based on the desired applications and flying needs and parameters. 


There is also growing interest to use UAS for traffic incident management. Some field tests 
demonstrated that UAS have unique advantages in providing instant traffic monitoring and 
data collection. However, many UAS research projects and field testing in this area are in the 
early stages of exploration and adoption. So far, no agency has made the use of UAS for 
traffic incident management a routine practice. 


Our review of the planning aspects of UAS for emergency preparedness and response shows 
that UAS can be very helpful in emergency and disaster management, but a uniform standard 
for UAS use across different areas (e.g., natural disaster, traffic incident, fire) is currently not 
available in Massachusetts. Therefore, we recommend that MassDOT consider the 
incorporation of UAS for emergency and disaster planning. Our review of UAS network 
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design shows that there are research efforts in academia that examined the architecture 
design and operation optimization of the UAS network. However, they are from a theoretical 
perspective and lack validation from implementation. 


Based on the outcome of the literature synthesis, arguably UAS have great potential for 
emergency and incident response. However, since many applications are still in the trial 
stage, the lessons learned from the literature synthesis are limited. Therefore, the research 
team recommends at this point, that it is important for Massachusetts to conduct a UAS trial 
program to study UAS’ potential as well as issues that may arise in practice. 


Finally, as summarized in Section 2.4, at a mid-stage of this project, a number of important 
questions were posed by MassDOT and MEMA focusing on how to develop a UAS 
emergency response network in the Commonwealth. This dialog was extremely helpful and 
helped the team to move from the broader literature review to a more refined and focused 
effort over the second half of the project. Limited by the scope of the project, we could not 
fully address all of the questions outlined in Section 2.4. Consequently, we focused on 
addressing two key questions below and leave the remaining ones for future research: 


1. What are the parameters for a good drone mission? When — in what situations — can 
drones be used? 

2. What are the design parameters of a UAS network for rapid response to emergency 
events in Massachusetts? Specifically, how many UAS stations are needed and where 
geographically to deploy them? 


These two questions will be addressed in the following section. 
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3.0 A UAS Network for Emergency Response in 
Massachusetts 


In this section, as stated previously, our research team focuses on two of the key questions 
raised by MassDOT and MEMA: 


1. What are the parameters for a good drone mission? When — in what situations — can 
drones be used? 

2. What are the design parameters of a UAS network for rapid response to emergency 
events in Massachusetts? Specifically, how many UAS stations are needed, and 
where should they be deployed geographically? 


In this second part of our research, our team undertakes two research tasks. First, we analyze 
historical traffic incident data to understand the features of incidents (e.g., frequency, 
location, and duration of various categories and subcategories of incidents) to address the 
first question of in what situations a UAS mission can be launched. Second, we then present 
an algorithm to decide the parameters of a UAS network for rapid emergency response 
within the Commonwealth. We developed the UAS networks for traffic incident response. 
We then show that the traffic incident-based networks can cover natural disasters as well, 
suggesting that the UAS networks can serve multiple purposes. 


One important advantage of the algorithm we develop is that it provides a framework that 
can be improved upon or refined in the future. Users can use it for different types of events, 
refine the parameter ranges (such as the radius of UAS stations and candidate UAS station 
locations) based on need, and optimize the design based on the objective or budget. This tool 
can be useful for different divisions in MassDOT and other states. 


3.1 Data 


3.1.1 Geospatial data 


3.1.1.1 Traffic incident data 

MassDOT collected the traffic incident data. This data is critical in answering both questions 
above regarding when UAS should be used and where severe incidents are likely to occur 
that need a UAS response team. This incident database provided a total of 73,224 traffic 
incidents across the Commonwealth over the period of April 2013 to October 2017. 
Individual incident data include the starting and ending time, location (latitude/longitude), 
type, and severity. The incidents are categorized by six event types: Planned Roadway, 
Roadway/Traffic, Fire, Environmental/Hazmat, Law Enforcement/Security, and 
Property/Structural Damage. The six types are further classified into 107 sub-types. They are 
also categorized into five severity levels: Daily Operations, Level 1, Level 2, Level 3, and 
Level 4, with the higher level indicating more severe events. 
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Figure 3.1: Illustration of historical traffic incident dataset (each point represents one incident 
and color indicates severity) 


3.1.1.2 Disaster Data 

Extreme weather-related events, such as damaging wind and flooding, are another aspect 
MassDOT and MEMA may want to consider when locating a rapid response UAS 
emergency network. While we readily admit historical location data on hurricane landfall in 
Massachusetts may not be the best predictor of future hurricane landfall, it could provide 
some useful information. The flood risk maps from the National Oceanic and Atmospheric 
Administration (NOAA) are a layer that may be useful, although future derivatives of what 
we implement here might want to consider richer data on road infrastructure elevation as it 
relates to these flood risk zones. 


In our analysis, four datasets with hurricane and flood information were utilized: 1) the 
influencing area of 1991 Hurricane Bob with peak winds > 50 knots (NOAA damaging 
winds category); 2) the influencing area of 1999 Hurricane Floyd with peak winds > 50 
knots; 3) NOAA 2% flood risk areas; 4) NOAA coastal flood risk areas. 


Note that a variety of hurricanes that hit Massachusetts over the last 30 years (e.g., Gloria- 
1985; Bob-1991; Floyd-1999; Beryl-2006; and Hanna-2008) were also investigated. The two 
hurricane events that had peak wind speeds over 50 knots (Bob and Floyd) were chosen, 
given NOAA defines “damaging winds” as ones exceeding 50 mph (1 knot = 1.15 mph). The 
hurricane wind speed spatial data layer had to be built from wind recording station data that 
was taken at stations in a variety of Massachusetts towns. Not all towns had such data, so 
Geographic Information System (GIS) interpolation was used to create a wind speed surface 
for both the Bob and Floyd layers. Figure 3.3 and Figure 3.4 shows the analyzing and 
interpolating process. In Figure 3.3 and Figure 3.4, the wind speeds are scaled by color, with 
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red indicating upper bound at 62-64 knots, green for the lower bound at 27-30 knots, and the 
transitioning colors indicate wind speeds in between . Towns with the upper bound speed (in 
red color) include Barnstable, Brewster, and New Bedford. Towns in intermediate speeds 
include Bourne, Edgartown, Milton, Plymouth, Taunton and Westfield. Towns with lower 
bound speed (in green color) include Beverly, Lawrence, Mount Washington, Nantucket, 
Norwood, Orange, Rockport, Salisbury, Southwick, Truro, Williamstown, and Worcester. 
Notably, Figure 3.4, most of the commonwealth is within the winds at 27-30 knots and the 
area at higher wind speed (62-64 knots) is in southern Massachusetts. The NOAA flood risk 
areas covered the entire state. 


Using GIS data conversion tools, each of the four weather disaster layers was converted into 
a raster with a standard 500x500 meter cell size, where each cell obtains an indicator 1 
(influenced) or 0 (not-influenced) that represents the status of the cell. 


a) 1991 Hurricane Bob influencing area _ b) 1999 Hurricane Floyd influencing area 


& 
¢ 


c) NOAA 2% flood risk areas | d) NOAA coastal flood risk areas. 


Figure 3.2: Illustration of the four weather-related disaster data layers 
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Note: Red: 62-64 knots, green: 27-30 knots. 


Figure 3.3: Towns with peak wind measurements for Hurricane Floyd (1999) 


Note: Red: 62-64 knots, green: 27-30 knots. 


Figure 3.4; Interpolated peak wind landscape for Hurricane Floyd (1999) 


3.1.1.3 Maintenance facilities 

We believe that the locations of MassDOT maintenance depots might be useful as potential 
UAS fleet deployment stations, given they are under the management of state DOT and could 
be easily upgraded to stage UAS supporting equipment, should MassDOT decide to purchase 
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and maintain their own fleet. There are 125 maintenance depots across the commonwealth; 
see the figure below. 


Lastly, the Massachusetts major roadway network GIS layer from MassGIS is used for 
visualizing the geo-location of all above-listed data. 


Figure 3.5: Illustration of MassDOT maintenance depot locations 


3.1.2 Features of traffic incidents 


In this subsection, we study the basic features of traffic incidents within the Commonwealth. 
This will help to answer the first question of when should UAS be used based on the event 
features, and also will help to identify historical traffic incident “hotspots” that would help 
locate UAS emergency response stations. We examine four aspects of the incidents, 
category (i.e., event type), frequency, severity, and duration. The aim is to reveal the most 
important features that are related to UAS missions. 


Regarding event type, incidents under different layers (categories/subcategories) are 
demonstrated respectively. Incident categories are very general groupings of types of 
incidents such as “roadway/traffic”, “planned roadway”, “property/structural”, “fire”, “law 
enforcement”, or “environmental or hazardous materials”. These broad categories are further 


broken down into more subcategories such as “road debris”, “potholes”, “construction 
event”, or “multi-vehicle accident (with or without injury).” 


In the figures and discussions that follow, we examine the types of categories and 
subcategories and their frequencies, as well as the level of severity of their incidents. The 
review is done with the intention of trying to understand what kinds of high severity 
incidents exist, the temporal duration of these incidents, as well as the geographic location of 
these incidents. This historical information will assist in determining possible geographic 
locations of UAS emergency response base locations. 
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Figure 3.6 demonstrates the frequency of each incident category. More than 90% of the 
overall incidents between April 2013 and October 2017 fall under the Roadway/Traffic or 
Planned Roadway categories. Property/Structural Damage and Fire are prevalent but occur 
much less frequently, while the last two categories related to law enforcement or 
environmental or hazardous materials are relatively rare. 


Roadway / Traffic 64.3% 


Planned Roadway 28.9% 
Property / Structural Damage 
Fire 


Law Enforcement / Security 


Environmental / Hazmat 
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Figure 3.6: Frequency of each category 


Figure 3.7 demonstrates the frequency of type of incident that occurs in the two most 
frequent incident broad categories (Roadway and Planned Roadway). Due to the limitation of 
figure size, only the top 10 most frequent incident subcategories are shown. Note that some 
of the most frequent incident subcategories are relatively minor. For example, the most 
prevalent subcategory, Debris/Rubbish, is likely not an event where a UAS network would 
need to be deployed to assess the situation. Similarly, pothole incidents, the fourth most 
prevalent in the dataset, also would not likely require UAS support. However, the 
subcategory MVA (Multi-Vehicle Accident) without injury or MVA with injury, while lower 
in terms of incident frequency, could be the kind of incident subcategory where a UAS 
response may be needed. 


Roadway / Traffic - debris / rubbish 

Roadway / Traffic - Disabled Motor Vehicle (DMV) 
Planned Roadway - Construction activity/event 
Roadway / Traffic - pothole 

Roadway / Traffic - dead animal 6.8% 

Planned Roadway - Other 

Roadway / Traffic - MVA without injury 
Roadway / Traffic - MVA with injury 
Planned Roadway - Bridge inspection 


Roadway / Traffic - trees / limbs 
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Figure 3.7; Frequency of top 10 incident subcategories under Roadway/Traffic or Planned 
Roadway 


Figure 3.8 turns to an examination of incident severity levels in the overall dataset. While 
Level 1 and Daily operations are the most common severity categories, Level 2 and Level 3 
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take smaller proportions though they are much more severe. Level 4 is very rare (only four 
Level 4 incident in the dataset). 


Level 1 60.5% 


26.2% 


Daily Operations 


Level 2 12.1% 


Level 3 1.2% 


Level 4j 0.0% 
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Figure 3.8: Proportion of each severity level 


In addition to the frequency, severity level is another important factor for consideration of 
UAS missions. Thus, a cross-analysis is further conducted to investigate and discuss the 
severity distribution of the incidents. 


Figure 3.9 shows the severity proportion of each category. The different colors indicate 
different severity levels: green for daily operations, light blue for Level 1, orange for Level 2, 
red for Level 3, and purple for Level 4. Most incidents in the fire category are at high 
severity levels (Level 3 or 4) despite the relatively low frequency. Regarding the two major 
categories, the Roadway/Traffic has a significant proportion of incidents at level 2 or higher 
severity, but the Planned Roadway incidents are mostly below level 2 severity, which 
suggests that this category may be one type where a UAS mission is not required. 


Roadway / Traffic | 64.3% 


Property / Structural Damage | | 5.0% 


Fire || 1.2% 
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Law Enforcement / Security | 0.3% Levelt 
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Figure 3.9: Severity proportion of each incident category 


Similarly, Figure 3.10 shows the severity distribution of each subcategory. As mentioned 
earlier, the top six most frequent subcategories mostly consist of less severe incidents (Daily 
Operations or Level 1). Thus, these events may not be worth launching a UAS mission. 
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Meanwhile, MVA without injury and MVA with injury subcategories show a high Level 2+ 
(i.e., Level 2 or above) proportion, suggesting that these subcategories are likely candidates 
for UAS missions. 
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Figure 3.10: Severity distribution of top 10 incident subcategory 


Another important feature of incidents is the event duration — the length of time from start to 
full operational cleanup of the incident — is another perspective that might help to answer the 
question of when and where a UAS mission should be flown. Figure 3.11 shows the overall 
duration distribution of incidents. We find that about 30% of incidents last over 1,000 
minutes (about 16.5 hours), and some last a relatively short time (e.g. 5 minutes). 


60% 
50% 
40% 
30% 
20% 


10% 


O, ~ u 
ia 0 7500 10000 12500 15000 17500 
duration (min) 


Figure 3.11: Overall duration distribution of incidents 
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3.2 Methods 


To develop a UAS network for emergency response, we follow two steps: 
e Step 1: to identify applicable incidents for UAS missions; 
e Step 2: to decide the parameters of the UAS network, particularly, the station number 
and station locations. 


These two steps will address the two questions raised above, respectively. Below, we 
elaborate on the two steps separately. 


3.2.1 Identify Applicable Incidents for UAS Missions 


In this subsection, we aim to identify events that are suitable for UAS missions. This will 
answer the Question 1 raised: “What are the parameters for a good drone mission? When — in 
what situations — can drones be used?” 


To this end, we consider two incident features, incident severity and duration!. The severity 
indicates whether an event is worth the efforts of flying a drone and the duration indicates 
whether it is feasible to use UAS. Specifically, for the severity filter, we consider events that 
are at severity level 2 or above. For the duration filter, we consider events the last between 30 
minutes to five hours (300 minutes). The reason is that for events that last shorter than 30 
minutes, the drones may not have enough time to prepare and respond; while for those lasting 
longer than five hours, drones may not be the most useful tools. In fact, it is found that most 
of the long-lasting events (>5 hours) fell into severity Level 1 or the level of Daily 
Operations. With the two filters, we identified the applicable incident set. The features of the 
applicable incidents (event type, frequency, severity, and duration) are shown on the next 


page. 


Figure 3.12 shows the frequency of each category for the applicable incidents. It is found that 
most applicable incident categories are roadway/traffic incidents, fire, and property/structural 
damage. Note that the distribution of incident types in the filtered set is quite different from 
the no filter case. 


1 For real-time application, this would be the predicted duration. 
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Figure 3.12: Frequency distribution of applicable incidents 


Figure 3.13 demonstrates the frequency of the top 10 subcategories with filters. Specifically, 
multi-vehicle accidents, roll-over, car-on-fire are the most frequent applicable incident 
subcategories. This result is consistent with the observations in the raw data statistic as 
shown above. 
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Figure 3.13: Frequency of top 10 subcategory of the applicable incidents 


Furthermore, Figure 3.14 shows the duration distribution with filters. Most incidents that are 
applicable for UAS response are shorter 400 minutes. 
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Figure 3.14: Duration distribution of the applicable incidents 


In short, to identify the applicable incidents, we consider two filters: 
e Severity level: level 2 or above. 
e Duration range: between 30 minutes to five hours. 


More filters can be further incorporated using the same procedure, such as the incident types. 
Notably, the setting of the filters will determine the applicable incident set, which should be 
further calibrated based on the operational features of the UAS, such as battery life and time 
needed to prepare missions, as well as the priority of the agency. 


3.2.2 Develop the Optimal UAS Station Algorithm 


In this section, we turn to determine the parameters of the optimal UAS network, which will 
address Question 2 above: “What are the design parameters of a UAS network for rapid 
response to emergency events in Massachusetts? Specifically, how many UAS stations are 
needed and where should they be deployed geographically?” 


The objective of the UAS network is to provide comprehensive coverage of incidents with 
the lowest possible cost. To this end, we make two assumptions: (1) a UAS emergency 
response station will cover a fixed radius geographic area with radius r; and (2) the cost for 
each station is a constant. Based on that, the problem can be reformulated to “how to use the 
minimum number of circles to cover the desired number of events?” This problem is a classic 
discrete K-center problem in mathematics known to be “NP-hard”? (39) and the optimal 
solution cannot be analytically found. To address this problem, we develop a greedy 
algorithm to solve it. Figure 3.15 shows the flow chart of the approach. The inputs to the 
algorithm are summarized in Table 3.1, including incident locations, candidate locations for 
stations, radius of UAS station, and expected coverage of the network. The pseudo code of 
the algorithm is provided in the appendix. An example of the inputs used in our analysis is 
also provided. 
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Generate the set for incidents covered by Generate the set for second-round 
second-round candidates candidates 


Rank second-round candidates by the Drop the candidates that the coverage 
number of incidents they cover does not meet a pre-set threshold 


Figure 3.15: Flow chart of the optimal UAS station greedy algorithm 


Table 3.1: Inputs of algorithm to optimize the UAS network 


Algorithm Inputs Example 
ch Incident points location The 5541 applicable incident points 
2 Candidate station location MA maintenance depot locations 
3 Radius of each UAS station r r= 5 miles 
4 Expected coverage of incidents | 95% 


? NP-hard problems are ones where there is no known polynomial algorithm. Consequently, the time it takes to 
find a solution grows exponentially depending on the scope or size of the problem. 
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3.3 Results 


3.3.1 UAS network for traffic incidents 


The proposed optimal UAS station algorithm is applied to the selected incidents with severity 
Level 2+ and duration in (30min, 300min), which results in a total of 5541 cases. The 
maintenance depots are used as the candidate locations for UAS stations. For the radius of 
UAS stations, we consider two different values, 5 miles and 10 miles. For expected coverage, 
we vary it from 90% to 95%. The summary of the inputs is provided in Table 3.2. 


Table 3.2: Inputs Used in UAS Network Optimization 


Algorithm Inputs Inputs Used In Analysis 
1 Incident points location The 5541 applicable incident points from filters 


(Severity level: level 2 or above; and Duration 
range: between 30min to 5 hours. ) 

2 Candidate station location MA maintenance depot locations 

3 Radius of each UAS station r r = 5 miles; r = 10 miles 

4 Expected coverage of incidents | 90%; 95% 


An example of the optimal UAS network is demonstrated in with a 10 mile radius and 95% 
coverage. Note that the incident frequency in each station is different. We define “key 
stations” as stations covering more than o incident per month during the observation period 
(between April 2013 and October 2017). The key stations are shown in red circles and the 
regular stations are in green circles. A total of 26 station locations are needed to cover 95% 
of the incidents, among which 13 are key stations that cover 81% of the incidents. It is also 
worth noting that the developed approach outputs show stable results despite selecting 
random starting points. The robustness is mainly in thanks to the two-stage method described 
in Figure 3.15 above (in the first stage, the raw candidates are generated; in the second stage, 
they are ranked and selected based on their importance). Another observation is that there are 
multiple incidents that occurred in very close time intervals (e.g. five minutes) at some key 
stations. Though such a situation does not happen frequently, it suggests the potential need 
for multiple UAS at some busy stations. 


Clearly, if we reduce the expected coverage, the network parameters will change. Figure 3.17 


shows the network with a 10 mile radius but 90% coverage. In this case, only 19 stations in 
total are needed, but there are still 13 key stations. 
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Notes: Red circles for key stations and green circles for regular stations. 


Figure 3.17: Optimal layout of UAS station (radius = 10 miles, expected coverage = 
90%) 
Apparently, the station radius is another important parameter that affects the optimal UAS 
station layout. Figure 3.18 shows the station layout and coverage when the station emergency 
response radius is reduced from 10 miles to five miles. With a smaller radius, the number of 
total stations required for coverage increase to 98, with 25 designated as key stations. The 
key stations now only cover 70% of the incidents. 
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Notes: Red circles for key stations and green circles for regular stations 


Figure 3.18: Optimal layout of UAS stations (station radius = 5 miles) 


We further conducted a sensitivity analysis regarding a different radius and expected 
coverage when keeping the other parameter unchanged. Figure 3.19 shows that the marginal 
cost (the number of stations per coverage) increases as the expected coverage approaches 
100% when the radius is kept the same. This suggests that the cost-effectiveness decreases 
when the expected coverage increases. Additionally, the marginal benefit of the radius 
decreases as the radius increases, as shown in Figure 3.20. This result suggests that 
increasing the radius will help to reduce the number of stations needed, but the the benefits 
decrease after a certain extent. The sensitivity analysis will help us decide what type of 
drones to use when given limited resources. 
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Figure 3.19: Number of stations in varied expected coverage (radius = 5 miles) 
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Figure 3.20: Number of stations in varied radius (expected coverage=95%) 


3.3.2 UAS station coverage on other disaster events 


In this section, we show that the UAS networks based on traffic incidents can be used for 
emergency responses to major weather-related events. These events are important as they can 
cause incidents such as downed trees or power lines, or major floods that might result in 
transportation infrastructure blockage. Specifically, two key major weather events in 
Massachusetts that causes these kinds of problems are hurricanes and other moisture-carrying 
events that cause major and rapid flooding to occur. 


3.3.2.1 Hurricanes 

Hurricanes have the obvious ability to cause extensive damage across Massachusetts. While 
we fully recognize historical data related to hurricanes are likely not a great predictor of 
future hurricane geographic footprints, it is a worthwhile exercise to examine the history of 
Massachusetts’ hurricane events to see how those footprints might influence UAS station 
placement. Considering the excellent reaction time and mobility capability of UAS, 
placement of UAS emergency response stations with some consideration of where hurricanes 
have hit before could enhance our ability to respond and could provide a first-hand 
understanding about areas where infrastructures was previously affected by damage (e.g. 
tree, power line falls or flooding) due to hurricanes. 


We considered two Massachusetts historical hurricanes — Bob (1991) and Floyd (1999) — as 
both had winds measured at 50 knots (about 57 mph). Next, we used the UAS network with a 
10-mile radius and 95% incident coverage developed based on traffic incidents to see how 
much the influenced areas of the hurricanes fall within the coverage of this UAS network. 
The visualized results are shown in Figure 3.21 and Figure 3.22 for Floyd 1999 and Bob 
1991 respectively, in which the dark shaded regions indicate the influenced area. Table 4-3 
shows the coverage result. 
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Table 3.3: Disaster coverage (radius=10, expected incident coverage =95%) 


Disaster Coverage Key station coverage 
Bob 1991 86% 71% 
Floyd 1999 94% 62% 


al w _ ; ) 
Mh. n 


Notes: Dark shaded region indicates influenced area of hurricane, red circles for key stations and green circles 
for regular stations. 


Figure 3.21: Station coverage on Floyd 1999 hurricane 


Notes: Dark shaded region indicates influenced area of hurricane, red circles for key stations and green circles 
for regular stations. 


Figure 3.22: Station coverage on Bob 1991 hurricane 
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Our historical analysis of hurricanes (50 knots or greater peak wind) reveals that the UAS 
stations have acceptable performances on assessing the disaster scenarios. The coverages on 
the investigated historical datasets all reach 80%+, even though they are not as thoroughly 
covered as what was discovered for roadway incidents. The likely reason for this is that 
hurricane high wind areas are found more closely to coastal areas, while many of the UAS 
stations are mostly arranged inland. 


3.3.2.2 Flooding 

Flooding events are another highly critical disaster type that could require UAS-based 
emergency response. To examine UAS emergency response station placement, we utilized 
NOAA 2% flood risk maps for the Commonwealth available at MassGIS. 


Figure 3.23 shows the UAS station coverage on NOAA 2% flood risk areas. The UAS 
stations provide an 88% total coverage with a 66% key station coverage. Note that the risk 
areas are shown in dots with their size representing the size of the area. However, since the 
areas are very small compared to the whole commonwealth, they appear like points in the 
figure. 
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Notes: Dots indicate risk area, red circles for key stations and green circles for regular stations. 
Figure 3.23: Station coverage on NOAA 2% flood risk area 
Overall, the results in this section suggest that a UAS network developed based on traffic 
incidents provide a very good coverage of the natural disasters. Of course, it is also possible 


to develop a UAS network based on the temporal and spatial distributions of the natural 
disasters. This can be easily done using the same procedure. 


38 


3.4 Summary 


In this chapter, the procedure to develop a UAS network for emergency responses was 
proposed, based on the analyses of multiple historical datasets. A framework was developed 
for designing an optimal UAS network, specifically to answer two major questions: 

1. 1) when — in what situations — can drones be used? and 

2. 2) where should UAS fleets be located? 


To answer the first question, the statistics of the Massachusetts incident dataset are analyzed. 
A simple and effective procedure is used to identify incidents for UAS application. Critical 
incidents are identified by two filters: a higher severity level (Level 2+), and appropriate 
duration (30-300 minutes). A greedy algorithm is developed to solve the second question, 
which automatically calculates the optimal parameter for the UAS network (i.e. the number 
and locations of the stations). The network parameters are discussed considering different 
station radius and expected coverage. Specifically, the number of stations needed increases 
with the expected coverage but decreases with the station radius. The further sensitivity 
analysis suggests the marginal benefits vary with the input range changes, which potentially 
impacts the cost-effectiveness. Furthermore, the UAS application on natural disaster 
scenarios is discussed with hurricane and flood information. The coverages on the disaster 
influenced areas are found to be satisfactory, even though they are not as thoroughly covered 
as what was discovered for roadway incidents. 
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4.0 Conclusions 


In this report, the applications of UAS for emergency response are examined through a 
thorough literature review. Moreover, we developed a flexible procedure to design a UAS 
network in Massachusetts and recommended the network design based on the traffic incident 
analysis. 


The literature review provides a comprehensive understanding on the public policy and 
administration aspects of UAS applications in the United States, particularly in 
Massachusetts. The findings also provide information on recent historical applications of 
UAS both nationally and internationally for emergency and incident response, as well as 
issues around the planning and design of UAS networks in advance of future emergencies or 
disasters. The practice and needs of UAS for emergency and incident response at MassDOT 
and MEMA were assessed. 


The framework of designing a UAS network for emergency response is proposed based on 
the analysis of multiple empirical geospatial datasets, including traffic incident data, natural 
disaster data, and maintenance depot locations. The features of the traffic incidents in a five- 
year period were investigated. A two-step method was developed to select the applicable 
incidents for UAS applications and optimally determine the UAS network parameters. The 
first step identifies the applicable incidents by the severity as well as duration filters. In the 
second step, a greedy algorithm was used to automatically determine the number of stations 
needed and the locations of the stations, based on the given radius of UAS stations and the 
expected coverage. The results from the developed algorithm indicate that the UAS network 
parameters vary with the UAS station radius and the expected coverage. Specifically, the 
marginal cost (the number of stations per coverage) increases as the expected coverage 
approaches 100% (radius is kept the same), suggesting that the cost-effectiveness decreases 
when the expected coverage increases. Additionally, the marginal benefit of the radius 
decreases as the radius increases, suggesting that increasing the radius will help to reduce the 
number of stations needed, but the the benefits decrease after a certain extent. Furthermore, 
the UAS network developed for the traffic incidents was able to provide a satisfying 
coverage on the influence area of several natural disasters. 


The results of the proposed framework can be used for decision-making, including UAS 
network parameters, drone types, fleet size, and applicable event type. It is worth noting that 
this effort could be conducted by considering that the constraint due to cost-related factors, 
e.g., budgets, cost-effectiveness, since the algorithm outputs (i.e. the number of locations of 
the stations) is closely related to the potential cost. 


Future work is desired to improve the network design. An obvious gap, and one intentionally 
descoped from this research, was the lack of any practical or operational considerations in the 
utility of drones for the incident types discussed. The next step is to think about the practical 
implications and implementation considerations of drones at these “key stations” and factor 
in important considerations such as drone flight times, sensor capabilities, and airspace and 
property flyover restrictions. For this purpose, a pilot study is desired to calibrate the 
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operational parameters of drones and identify potential issues that may arise in practice. 
Another future direction is further optimize the UAS network by considering more aspects of 
cost, such as the cost of the drones (capture purchase), drone maintenance, facility operation 
(such as the trucks needed to carry the drones), etc. 
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5.0 Appendices 


Appendix A: Existing Applications of UAS in 
Emergency Responses 


This part presents the detailed review of the existing applications of UAS in emergency 
responses. It focuses on how UAS can be used after an emergency has occurred. 


Fire 

Compared with manned aircraft, UAS have proven to be a cost-effective solution in response 
to small-to-medium scale wildfires (40). Helicopters cost roughly $2,500 per hour to fly, 
while UAV cost a fraction of that amount (41). In terms of efficiency, multiple UAV can be 
configured to perform rescue tasks simultaneously when a building is on fire (42). 

Fire management is probably the most popular application area of UAS compared to other 
disasters. Between 2006 and 2010, the U.S. Forest Service and the National Aeronautics and 
Space Administration (NASA) flew 14 missions with NASA’s Ikhana drone over 57 fires in 
the U.S., using a multispectral sensor to provide intelligence to firefighting teams (43). The 
Fire and Rescue Department of Montgomery County in Maryland purchased three drone 
systems in 2014 to provide real-time imagery during high-rise blazes to show fire strength 
and to assess building structural integrity (43). 


UAS can be very useful during building fire rescue. A British start-up, Unmanned Life, 
developed a software package to simultaneously operate multiple autonomous drones to 
assist with a fire rescue mission (42). UAS can also be used for pre-disaster warning. Seo et 
al. (44) proposed a monitoring and emergency response method utilizing UAS for fires in 
buildings. This system can detect building fires using indoor and outdoor UAS and help to 
safely evacuate buildings. 


Earthquake 

Given damaged surface transportation infrastructure after an earthquake, UAS can be very 
useful in mapping the affected area to assess damage or share rescue sources (40). UAV can 
provide unique viewing angles at low altitudes, which is not possible from manned aircraft 
(43). 


UAS are frequently used to perform mapping and rescue missions after an earthquake. In 
2013, as part of the reconstruction efforts following the 2010 earthquake in Haiti, the inter- 
governmental International Organization for Migration teamed up with the Swiss nonprofit 
Drone Adventures to assess destroyed houses, take a census of public buildings and 
hospitals, and monitor camps for internally displaced persons (45). 


Flooding 

UAS have proven to be very effective in conducting pre-disaster and post-disaster mapping 
and assessment. UAV can be deployed to monitor the conditions of dams and river banks. In 
case of a broken dam or in situations where citizens are trapped in flooded areas, UAS can 
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support search and rescue activities (40). In 2013, a mapping drone called eBees was 
deployed to identify potential flood-prone zones in Haiti, and helped collect information for 
constructing protective infrastructure to avoid future catastrophes (45). 


Hurricane 

After a hurricane, UAS can be deployed to affected locations which are often inaccessible 
due to fallen trees or utility poles, flooding, road damage, etc. (46), allowing responders to 
gain a Clear picture of the overall damage distribution and to then prioritize rescue tasks (43). 


UAS were used in Hawaii in 2014 following Tropical Storm Iselle (46). They were 
integrated into the Mobile Emergency Response and Command Interface (MERCI), a system 
for collecting damage assessment data. Drones were equipped with cameras and sensors to 
collect data to provide situational awareness and for identifying and quantifying damages. 
The data collected by drones was then integrated with other data sources (such as crowd- 
sourcing data), which were found invaluable for rescue, response, and recovery. A main 
challenge lies in how to rapidly process the various data sources and extract useful 
information. The “Flying COW”—a drone used in AT&T’s Network Disaster Recovery 
system—was used to provide emergency 4G coverage in Puerto Rico during the aftermath of 
Hurricane Maria. Each Flying COW was able to cover about 22 square miles (42, 47). 


Outside of the U.S., Drone Adventures used eBees to assist the Philippines with a post- 
disaster needs assessment following the 2013 Typhoon Haiyan. The gathered data provided 
local leaders and humanitarian organizations with detailed maps and damage information to 
plan and coordinate relief efforts (45). Drones were also used for post-disaster assessment 
after tropical Cyclone Pam in 2015 in Vanuatu (48). Ezequiel et al. (49) described an 
example of using ArduPilot MegaMission Planner to process UAS flight data on a coastal 
section of the City of Tacloban in the Philippines. 


Volcanos 

Amici et al. (50) developed a UAS-based real-time data acquisition system and tested it at 
the Le Salinelle mud volcano located at the southwest boundary of the Mt. Etna volcano, by 
integrating a low-weight thermal camera into a hexacopter. In their test, the UAS was able to 
record and transmit real-time videos to the remote ground station under an extreme 
environment. 


Landslides 

Huang et al. (51) developed a highway landslide warning and emergency response system 
based on UAS. This system consists of two main components: a landslide warning sub- 
system and a UAS emergency response sub-system. 


Search and Rescue 

The first 72 hours after a disaster are the most precious for search and rescue operations (52). 
However, search and rescue operations in this period can be better suited for UAS than 
manned aircrafts (43). Different types of UAV can be applied in a variety of scenarios. A 
simple application is to use UAV to provide live video feeds that will assist in searching for 
missing persons. 
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In 2012, the Texas search and rescue group EquuSearch was able to use a drone to locate a 
missing boy in a pond after a failed ground search. The ground search was challenged by the 
presence of alligators, wild hogs, and poisonous snakes (43). In 2014, amateur drone operator 
David Lesh located an 82-year-old man alive in a bean field using an octocopter. Prior to 
this, hundreds of volunteers and a helicopter team had spent several days trying to find him 
(43). 

While small drones excel at locating people such as in the two cases above, heavy-lift drones 
are often equipped with the additional capability to transport personnel and rescue survivors 
from a disaster site (43). 
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Appendix B: Incident Statistics with 
Different Thresholds 
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Figure 5.1: Frequency of each category under different thresholds 
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Figure 5.2: Frequency of top 10 subcategory under different thresholds 
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Figure 5.3: Duration distribution under different thresholds 
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Appendix C: Pseudo Code of the Greedy 
Algorithm 


The pseudo codes of the greedy algorithm are shown in Table 5.1. 


Table 5.1: The greedy algorithm pseudo codes 


Step # Description 


Input Incident set I = {1,, Ip, ..., [,}, Station candidate position set C = 
{C,, Cz, ..., Cm}, Station radius r, the expected coverage E 
Step 1. Cover incidents by an appropriate amount of second-round candidates 


Covered incident set CI = @ 

Second-round candidates set SC = @ 

Pick a random start point C* in C 

While size(CI) < n and size(SC) < m: 

J* =Find the incidents in J that are covered by C* with radius r 
Cl.append(/*) 

I. remove(/*) 

SC.append(C*) 

C* = find the furthest candidate position from C* in C 


Step 2 Select final stations from the second-round candidates set by the number of 
incidents they cover 
SC.rank(by the number of incidents in IJ that are covered by SC; with radius 
r) 
Optimal station set S = @ 
For potential stations SC; in SC: 
II* =the number of incidents in IJ that are covered by SC; 
If coverage > E: 
Break 
S.append(SC;) 
II.remove(I1") 


Output Optimal station set $ 


In the greedy algorithm described above in Table 5.1, Step 1 is designed to avoid over- 
computation regarding the incident and candidate source (redundant candidates are dropped 
as soon as all incidents are covered, and vice versa). Step 2 is employed to guarantee the 
robustness by ranking the initial set by importance. The final station set is then selected from 
the more relevant ones to less relevant ones until meeting the expected coverage. This page 
left blank intentionally. 
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Executive Summary 


This study of Assessment of Unmanned Aircraft System Situational Awareness Technology to 
Support Applications in Surface Transportation was conducted as part of the Massachusetts 
Department of Transportation (MassDOT) Research Program. This program is funded with 
Federal Highway Administration (FHWA) State Planning and Research (SPR) funds. Through 
this program, applied research is conducted on topics of importance to the Commonwealth of 
Massachusetts transportation agencies. 


The rapid proliferation of UAS (unmanned aircraft systems also known as drones) creates 
serious challenges to transportation facilities and traveling public. Designed to expand and 
update the results of the Phase I Counter-Unmanned Aircraft System (CUAS) study also funded 
through MassDOT with FHWA monies, the Phase II study presented herein was conducted to 
review the current technologies available to detect, track, and identify small UAS entering 
restricted airspace, specifically near critical ground transportation infrastructure, including that 
located within densely populated metropolitan areas. 


The objectives of this research were: (1) to accomplish a UAS-related literature synthesis of 
commercially available counter-drone technologies; and (2) to design a prototype of a field test 
to evaluate CUAS products intended to detect, track and identify cooperative and non- 
cooperative drones in the vicinity of critical transportation facilities; and (3) to validate the 
prototype and to conduct a field test of select CUAS products and technologies. Due to a number 
of logistical constraints, rigorous field testing of CUAS technologies was not conducted during 
this study. 


The research found that CUAS systems that integrate multiple technologies to detect, track, and 
identify UAS are the most promising solutions for protecting critical transportation 
infrastructure. In addition, the selection of CUAS products should consider all environmental 
factors around the protected facilities that can dramatically affect the performance of tested 
products, such as terrain, weather, noise, among others. Finally, the decision to implement a 
specific CUAS product should take into account the current regulatory framework in the U.S. 
and restrictions specific to a given state and jurisdiction. 


vii 


This page left blank intentionally. 


viii 


Table of Contents 


Technical Report Document Page .............ssccssccssscssesessccseccesccesecscnsscseccssccesscsenssesacessccessessnnsenacessacs i 
PLChMO WCU ECTS: oa tip rd ce nccsei nies olde: cess ace eedtarnedeGie anes ides des svacandies ia SGovescenidcdec set tatadbarceeGs weaniens V 
DUS VATA frescos race ea hcec sso idan de secs chu ea BO ed ees ds eed Sess asc A Dal eaecagdderaa eS aed ea dened OR Vv 
ERS CUti VG SUNIL Y sg gedes cas ac oscarveanscaryats cacianl co uetub cde sewebynsaiaedes an gueoi anne dutdaiaarmvenssugtacn eit eeiec ane vii 
EA DEE COIN INOS oes ard cry st ces Gevee angen vSesweinesianry cy euesonures nat sa ep ebei tens danse exeseaebnedoa ene tesa Mmeased bean cb san rh ix 
List: Of Tables: si. j3sscsssviacissiestetiarsaseethesiiviatviadeasiasietaseisaveatiat eaeatavaradiatismeelo inertia xi 
EASE OE PUG UL OS sci cance esascesevzactuecee st aayeteaviaacoaes cu ctutev ues ceaewsa casein cept de taney eee ata raaneaaeete cd xiii 
BTS EOL Px COOL YTS «oes Seether te Od le tel aula ede aal Ss, xv 
TiO WtPO MUCHO s.cassgsasSaniesscrnsedneansdug bonainuaiivun ony dacaubnignavadeh) oasauivon eur anawSnivearddel duagabiguaslaawneuasielvealoe 1 
Ly LsPPODIGTSSLALCECNIE, 33.8 isis i.s'5y nests scensbeigasicon ips Vustscii ue uasbos id sesurightivesiessaad nsioyadesbods othastodasusheutladespoateagosdaies 1 
122; Research ODjectiVes dvseeisiciesecttiaus vesceudeatavnalanct ade niente Gein eer aeiave teen 1 
Ld Report. Outlines, asctiecss sea. tetstatacieeass deca setts einer eee cd dah sees ene ae hea ceeeee Tea ees 2 

2 REGGAE VICTMOCUOLO DY. 9 state fecal ocaiotba dec alan aur ta tealat Dodmany fee Sakae anaes odes Mela talee Mes ontimute 3 
2.1 Literature Synthesis and Preliminary Evaluation .............ecescsssssceeecsseesceeecesececeeecesecsaceaceeesaeeaeeeneeaes 3 
2.2 Design of the Prototype of the Field Test................cssccosssscssesecteccenecnetcosensenceeossetcesensetensensenessoseasesees 4 
Be RESULTS cocsscithaxhcatsaohesus Siepeadusnalduetacdedtedsonsclee Gasatued ateanduntiant obkogeonesuteneetinudep wand sue bandlobnaseaneduedacpeunareancs 5 
SALI ratve?S yItheOsisS:..viceescsetesiek ei cesses ccdsetsedusoatesavdeencestudauasssidcoeh sivedaasevitessesYovagabsvdlebasteanteadecldicasHeoctaesst fs) 
3.2 Preliminary Evaluation of CUAS Products «0.0.0... cceeccssssscesscesecseeecsesecsesencesecsaceacecesaecaeeeneeseeseeeeeenees 7 
3.3 Design of the Prototype of the Field Test... eeeescssecseesecesecseseecesecsaecaeeeceaecsesenesecsaeeaeeseesaeeneees 10 
Dy Se LC TeSt-Site: SCLC CON savas seek seek res hantans eevee ccwetaoncan dees odauvabin tod ebbaen banceas area bocanta teow eg eb omeanebuaan ee pak 12 
3.3.2 Additional Details for Consideration. ............ecessessesceecceseceeeeecesecsaeeaeecesaecaeeeneesecseeeneeeeenaeeatees 13 

3.4 Field Testing and Demonstrations ............ccssccssccssecsseceseceecesecessceeseceneceneeeseecssecsuecsaecsaeceseceeeeeeeeeneeees 14 
4.0 Conclusions and Recommendations ...............:cssscessessceseessseeeceescesecsseeseesoeseeceesceseeeseeseeeneseeseens 15 
Op ROTEIRNCOS vaapia svencraisect sean emtcighe aan peacaunednde abliieoaudane Mas eprarecaiah deaehtsn ems aehete aad daeeip ahead mes 17 
GO A PP OnCiGes cyscesst csachectis cating sbcsaleepessuneveiaesiutecevtececantia)s eeoeestectevagstadielasavee atic avid eaten eet 19 
Appendix A: UAS Detection and Tracking Systems ............escsssesceeeceseeseeeeeesecneeeceesecseeeaeeseesaeeneeeneeaes 19 
Appendix B: UAS Interception and Interdiction SysteMs............ceeescsssesceeeceseceeeecesecaeeeceeeceaeeaeeeneeaes 22 
Appendix C: UAS Detection, Tracking and Interdiction Hybrid Systems... ceeeeeseeseeeeeeecseeeeeeeees 25 
Appendix D: Legal Barriers to CUAS Operations ............scsesssecsseeceeeceseeateeceesecseeeecesecsaeeaceseeaeeaeeeneeaes 27 


ix 


This page left blank intentionally. 


List of Tables 


Table 3.1. UAS detection, tracking, and identification technologies ..............csceeeeeeseeereeeeeees 6 
Table 3.2. sUAS detection, tracking and identification systems recommended for field test 10 
Table 3.3. Major benefits and challenges of CUAS field test Options............ccceesseeeteeeteeeees 11 
Table 3.4. CUAS technology performance indicators ............cceeseeseceteceseceeeeeeeeeeaeecseecneeeees 13 
Table 6.1. UAS detection and tracking systems: U.S. manufactures ............:ccsceesseeeteeeeeeeees 19 
Table 6.2. UAS detection and tracking systems: Manufacturers outside U.S. ........eeseeeeeees 20 
Table 6.3. CUAS detection, tracking and identification product evaluation... ZA 
Table 6.4. UAS deterrence and interception methods: Pros and COMS............::cseesseeeeteeeeeeeees 2a 
Table 6.5. UAS interdiction systems: U.S. manufacturers ...........ceceseesesseceeceeeenceeeeeseeeaeenaee 23 
Table 6.6. UAS interdiction systems: Manufacturers outside U.S. ........cecceceseeseceeeeeseeeeeeeees 24 


Table 6.7. UAS detection, tracking, and interdiction hybrid systems: U.S. manufacturers ... 25 
Table 6.8. UAS detection, tracking, and interdiction hybrid systems: Manufacturers outside U.S. 
sy cee eis ea bb av aeons Caesar ese a nue eavea aaa boa esce dade Saw Seacpus cahedibee tips td das seapevinegarbereen ohne 26 


xi 


This page left blank intentionally. 


Xil 


List of Figures 


Figure 3.1. Typical detection ranges of various CUAS technologies vs. UAS flight time ...... 


xiii 


This page left blank intentionally. 


xiv 


List of Acronyms 


Acronym Expansion 

ADS-B Automatic Dependent Surveillance—Broadcast 
ADS-R Automatic Dependent Surveillance—Receive 

Al Artificial Intelligence 

AUDS Anti-UAS Defense System 

COTS Commercial Off-The-Shelf 

CUAS Counter-Unmanned Aerial System (Technology) 
EO Electro-Optical 

FAA Federal Aviation Administration 

FHWA Federal Highway Administration 

FTA Federal Transit Administration 

FRA Federal Railroad Administration 

GPS Global Positioning System 

GNSS Global Navigation Satellite System 

IR Infra-Red 

LiDAR Light Detection and Ranging 

MassDOT Massachusetts Department of Transportation 
NextGen Next Generation Air Transportation System 

RF Radio Frequency 

UAS* Unmanned Aerial System(s) or Unmanned Aircraft System(s) 
sUAS? Small Unmanned Aerial System(s) or Small Unmanned Aircraft System(s) 


* The terms “UAS” and “drone” are used interchangeably. 
? Per current FAA regulations (Pub. L. 112-95, sec. 331(6)), a SUAS is defined as a small unmanned aircraft 
(system) with a takeoff weight under 55 pounds. 
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1.0 Introduction 


This research project, on the Assessment of Unmanned Aircraft System Situational Awareness 
Technology to Support Applications in Surface Transportation, was undertaken as part of the 
Massachusetts Department of Transportation (MassDOT) Research Program. This program is 
funded with Federal Highway Administration (FHWA) State Planning and Research (SPR) 
funds. Through this program, applied research is conducted on topics of importance to the 
Commonwealth of Massachusetts transportation agencies. 


1.1 Problem Statement 


Cost reductions and innovations in global positioning systems (GPS), cameras, and other 
advanced sensor-based technologies have led to increased use of small unmanned aircraft 
systems (SUAS). The Federal Aviation Administration (FAA) estimates that combined hobbyist 
and commercial sUAS sales will rise from 2.5 million units in 2016 to 7 million units by 2020 
(1). Another report predicts that there could be more than 2.5 million sUAS in the United States 
by 2020 with a takeoff weight, over 0.55lbs, for which current FAA regulations require FAA 
registration (2). FAA regulations define a SUAS as a small unmanned aerial aircraft with a 
takeoff weight under 55 pounds. 


Though it promises new opportunities, the rapid proliferation of sUAS also has the potential to 
lead to activities that may harm people and destroy or damage property. In order to ensure public 
safety and security, there is a need to evaluate technologies that can detect, track, and identify 
cooperating and non-cooperating sUAS, specifically those operating near sensitive areas such as 
transportation infrastructure. 


1.2 Research Objectives 


The objectives of this research were: 


1. To create a literature synthesis which focuses on CUAS technologies capable of 
detecting, tracking, and identifying sUAS near critical surface transportation 
infrastructure. 

2. To develop a prototype of a pilot study for field testing CUAS technologies. 

3. To conduct a field test to evaluate selected CUAS products, and to provide 
recommendations to MassDOT regarding potential CUAS solutions to address problems 
related to non-cooperative UAS near critical transportation infrastructure. 


1.3 Report Outline 


The remaining sections of this report are organized as follows. Chapter 2 describes the research 
methodology for this project. Chapter 3 describes the results of the literature synthesis and the 
initial COTS (commercial off-the-shelf) product selection, provides an outline of the prototype 
for a field test to evaluate select CUAS technologies, and discusses the observations made during 
the field demonstration of selected CUAS. Chapter 4 gives conclusions and recommendations. 
Chapter 5 provides the list of references used in this study. Chapter 6 contains appendices with 
detailed data collected during the literature synthesis, as well as other reference material, 
including details on the COTS CUAS technologies and products considered during this research. 


2.0 Research Methodology 


The first task of this project was to conduct a literature synthesis to identify the CUAS 
technologies available worldwide. The results of this task were presented in the Technical 
Memorandum on Literature Synthesis. 


The second task of the project was to perform a preliminary selection of CUAS technologies 
based on both technical parameters and some non-technical characteristics that were also found 
to be important during the literature synthesis. 


The third task of this project was to develop a prototype of a pilot study for a field evaluation of 
select CUAS technologies. 


The fourth task of the project was to execute field testing of the selected CUAS technologies. 


The first two tasks cover the first research objective presented in Section 1.2; the third task 
covers the second research objective; and the last task covers the third research objective. 


2.1 Literature Synthesis and Preliminary 
Evaluation 


The purpose of the literature synthesis was to collect preliminary information about technologies, 
trends, manufacturers, and products that can help to detect, track, and identify both cooperative 
and non-cooperative UAS in the proximity of critical transportation infrastructure. The 
information was collected from professional literature, internet publications, the Transportation 
Research International Documentation (TRID) database, conference proceedings, manufacturer 
brochures, and other sources. The information gathered through the literature study was also 
used to perform an initial selection of the commercial off-the-shelf (COTS) CUAS products. 


The evaluation of CUAS technologies included three steps as described below: 


e Step 1: Initial selection of CUAS products on the basis of: (a) U.S. market availability; 
and (b) compliance with U.S. civilian regulations. 

e Step 2: Evaluation of selected CUAS in terms of technical parameters and capabilities. 

e Step 3: Field testing of select CUAS products identified during the technical evaluation in 
Step 2. 


The literature synthesis covers Task 1 while the evaluation of CUAS covers Tasks 2 and 4. 


2.2 Design of the Prototype of the Field Test 


A prototype for the field testing of CUAS technologies was developed on the basis of the 
literature synthesis. The field test was intended to evaluate selected COTS CUAS technologies 
capable of detecting, tracking, and identifying cooperating and non-cooperating UAS, including 
1) assessing their performance, capabilities, and reliability, as well as 2) evaluating their practical 
utility for protecting critical surface transportation infrastructure. 


The research team designed the prototype of the field test to be similar to experiments conducted 
by the military (3) and cross-government research organizations (4), with the major differences 
related to the special focus on protection of ground transportation infrastructure, limited scope 
(detection, tracking and identification technologies only) timeframe (a few days) and budget. 


The exercise can be roughly described as a “war game” between the individual or a group of 
counter-UAS manufacturers who will try to protect a designated facility (“defenders”) and an 
individual or a joint group of MassDOT/UMass UAS pilots, equipped with a variety of different 
airframes, who will try to access the designated facility (“attackers”). 


The field tests have been designed to allow maximum flexibility to vendors, MassDOT, and the 
research team. This means, for example, that the test may be conducted at different times and 
different locations for different vendors or products. 


The design for the field testing includes six distinct options. Those options include the following: 


1. Each selected CUAS product will be tested at a single location; 

2. All selected products that utilize the same type of technology (e.g. radar) will be tested at 
a single location; 

3. All selected products with different technologies will be tested at a single location; 

4. Each selected CUAS product will be tested at multiple locations; 

5. All selected products that utilize the same technology (e.g. radar) will be tested at 
multiple locations; 

6. All selected products with different technologies will be tested at multiple locations. 


3.0 Results 


3.1 Literature Synthesis 


During the literature synthesis, the research team identified 49 different COTS CUAS products 
from 30 manufacturers. Twenty-four of these products are available on domestic market, the rest 
are sold only outside the U.S. 


A number of CUAS technologies that detect, track, and identify cooperating and non-cooperating 
sUAS have emerged in recent years to help protect public safety and critical transportation 
infrastructure. Some technologies initially developed for military applications may not be 
suitable for civil applications due to their high costs or because they may introduce additional 
hazards, disrupt the normal operation of critical communication and navigation equipment, or 
raise privacy and health-related concerns. Both MassDOT and FHWA are interested in finding 
the most appropriate technological solutions to address potential sUAS-related threats to critical 
transportation infrastructure while minimizing potential negative impacts associated with the use 
of CUAS technology The literature synthesis presented in this section is intended to assist 
MassDOT and the FHWA in better understanding the current state of the practice of CUAS 
technology. 


Building on the 2016 Phase I review of the commercial off-the-shelf (COTS) CUAS products 
and survey of a diverse group of decision makers (5), the research team conducted a literature 
synthesis to identify currently available technologies that can detect, track, and identify UAS 
entering restricted airspace, specifically near critical transportation infrastructure. This study is 
wider in scope than the Phase I project, and focused on protecting a wider variety of surface 
transportation facilities beyond airports. Therefore, the literature synthesis was expanded to 
include technologies that can protect smaller yet equally important ground transportation 
infrastructure including within densely populated urban areas. 


There has been a dramatic change in the landscape of CUAS technologies since the Phase I 
study. The numbers of both manufacturers and available products have quadrupled. As the 
market has become more saturated, some less competitive products and manufacturers have left 
the market, while others have merged efforts with former competitors or large, diverse 
electronics and defense industry consortia. 


While the research team tried to examine all existing technologies, the Phase II synthesis focused 
on civilian off-the-shelf CUAS products commercially available worldwide. UAS detection and 
tracking technology solutions can be divided into two groups. The first group consists of devices 
that utilize active detection methods. The second group includes devices that utilize passive 
detection methods. The most common active detection method is radar, which emits signals in 
the radio frequency (RF) spectrum and then captures the signal reflection from the aircraft and 
other moving or static objects. Passive detection methods utilize electro-optical, acoustic, and RF 
sensors to capture signals emitted by the aircraft itself. 


Both passive and active systems have proven to be effective in detecting and tracking UAS at 
both long (radar and RF-spectrum scanning) and medium to short distances (electro-optical, 
acoustic) (5). A brief summary of the advantages and drawbacks associated with different 
detection, tracking, and identification technologies is presented in Table 3.1. 


Table 3.1. UAS detection, tracking, and identification technologies 


Active Passive 
Radar Radio Frequency Electro-Optical Acoustic 
Advantages | Long-range; Long-range; High accuracy of Low cost; 
all-weather all-weather; tracking and high accuracy 
ability to track pilot | identification, of tracking 
and UAS including the payload 
Drawbacks | Affected by terrain; | Can’t “see” UAS Limited range; Limited range; 
limited UAS flying in a fully affected by elements | affected by 
identification autonomous mode and terrain noisy 
capabilities environment 


As shown in Table 3.1, there is no single universal solution for UAS-imposed threats. There are 
a number of reasons for this. The primary one is that UAS are typically small targets that may 
have a wide variety of physical characteristics and that are usually moving at low altitudes. Also, 
sUAS are usually made of composite materials that decrease the probability of stable and reliable 
detection, tracking, and identification with radar technology. In addition, sUAS do not carry a 
transponder such as the one used in the Automatic Dependent Surveillance-Broadcast (ADS-B) 
systems proposed in the Next Generation Air Transportation System (NextGen) for aircraft in 
controlled airspace. Moreover, there is currently no single standard for sUAS communication 
protocol or a specific frequency band. However, there is hope that this will soon change. DJI, the 
largest manufacturer of commercial UAS, just announced that all its drones that require FAA 
registration are going to be equipped with the ADS-R receiver starting in 2020; this is seen as a 
first step toward integrating sUAS into the national airspace (6). Finally, there is a growing trend 
of utilizing a wide group of stakeholders in the establishing up of standards on sUAS electronic 
communication and identification procedures (7). The push towards active ID mechanisms could 
establish a common feature that would enable a fairly universal detection, tracking, and 
identification approach. The effectiveness of potential solutions depends on how the standard ID 
is implemented. However, challenges still remain with UAS that operate in radio-silent mode. 


In order to achieve the most reliable performance, the majority of sUAS detection and tracking 
systems must integrate multiple types of sensor technologies, both active and passive. Examples 
of such comprehensive solutions are currently offered by SRC, Inc. (Gryphon Skylight, ACR 
Hawk) and by Dedrone (DroneTracker Multi-Sensor). The smaller DroneTracker system offers a 
range of UAS detection and tracking of up to 500 meters (1,640 ft.), while the larger Gryphon 
Skylight claims the capability to detect UAS as far away as 10 kilometers (6.1 mi.) with radar, 
and up to 3 kilometers (1.9 mi.) with its spectrum sensing and slew-to-cue camera (8, 9). 


Another notable comprehensive sUAS detection and tracking system is offered by DeTect Inc. 
The DeTect DroneWatcher equipped with HARRIER Drone Surveillance Radar provides a 
comprehensive, layered solution for detection, tracking, alerting, and interdiction of DJI 
Phantom-size UAS at distances of up to 4 kilometers (3.1 mi.). Advanced technology combines 


Signals Intelligence (SIGINT) which gathers information by intercepting transmitted signals, and 
radar for detection and tracking (10). Often, higher-end drone detection and tracking systems can 
also integrate and control third-party devices including signal jammers to intercept non- 
cooperative intruder sUAS. However, the price of such systems is often outside the budgets of 
smaller transportation facilities and operators. In addition, there are legal restrictions which limit 
wider implementation of such devices in the U.S. 


It is worth noting that on the low end of the CUAS market, there are a number of innovative 
products that are either free or very inexpensive. Such products include apps that can turn a 
WiFi-capable consumer electronic device - such as a smartphone, tablet, or computer - into a 
personal UAS detector. 


3.2 Preliminary Evaluation of CUAS Products 


Evaluation and selection of sUAS detection, tracking and identification systems is not a trivial 
task for a number of reasons. First, there are numerous variables to consider related to 
operational environment, potential vulnerabilities, types of target sUAS, capital and operational 
costs, among others. Operational environment-related variables may include the landscape, 
prevailing weather, and population density near a protected facility, among others. Potential 
vulnerabilities will vary with the type of the facility. The type of UAS as well as its size will 
greatly impact a CUAS system’s ability to detect, track and identify intruders. Capital and 
operational costs will affect the ability of the transportation facility managers to provide 
sufficient level of protection for their facilities. Degree of compliance with federal, state, and 
local laws and regulations will greatly affect potential level of implementation of CUAS 
technology. Finally, concerns associated with potential collateral damage may significantly 
restrict CUAS adoption under certain conditions. 


The evaluation of CUAS technologies was completed in the following three steps. 


Step 1: Initial selection of CUAS products on the basis of: (a) U.S. market availability; and (b) 
compliance with U.S. civilian regulations. 


Step 2: Evaluation of selected CUAS in terms of technical parameters and capabilities. 


Step 3: The study proposed field testing of select CUAS products chosen on the basis of the 
evaluation in Step 2. It was expected that the field testing will be conducted by MassDOT and 
the UMass research team at a location and using a testing format selected by MassDOT. Due to a 
number of logistical constraints, rigorous field testing of CUAS technologies was not conducted 
during this study. The constraints against field testing included the regulatory restrictions in the 
U.S., lack of time to finalize the format of the field tests, limited funding, and the challenges of 
trying to have multiple vendors participate in a single field test. In lieu of field testing, there were 
a number of field demonstrations conducted by individual CUAS vendors and attended by 
MassDOT staff and others. Those demonstrations did not include rigorous testing of CUAS 
products under a variety of conditions, but the results of the demonstrations were still 


informative. MassDOT may consider incorporating field testing into a future round of CUAS 
research. 


On the basis of the findings of the Phase 1 study and the past UAS experiences of MassDOT 
staff and the UMass research team, a decision was made to group the CUAS evaluation 
parameters into three categories: 1) primary performance-related; 2) secondary performance- 
related; 3) other important parameters such as capital and operating costs as well as regulatory 
constraints. As suggested by the panel of experts who contributed to the Phase I CUAS review 
and by the feedback from MassDOT on the draft Phase II literature synthesis, each category of 
parameters was evaluated independently. 


The primary performance-related parameters include detection, tracking, and identification 
ranges. Those parameters are paramount for the successful protection of transportation facilities 
as they directly affect amount of time available for authorities responsible for the facility 
operations to select and apply appropriate countermeasures. The secondary performance 
parameters include the ability to detect and identify payload, operate in adverse conditions, and 
detect rogue drones that operate in a fully-automated, radio-silent mode. The last category 
includes other important non-performance related parameters such as system capital and 
operational cost; regulatory compliance; as well as parameters related to collateral damage or 
potential environmental impacts. Figure 3.1 demonstrates the importance of the detection, 
tracking and identification ranges for the successful protection of critical infrastructure, and also 
provides a glimpse of challenges associated with such tasks. 


The horizontal axis of the graph presented in Figure 3.1 indicates the distance from the drone to 
the protected area. The icons below the horizontal axis provide the typical detection ranges for a 
radar-based system (approximately 7.5 mi. or 12km.), an electro-optical (EO) system 
(approximately 2.5 mi. or 4 km.) and acoustic detectors (approximately 0.5 mi. or 0.8 km.). The 
vertical axis of the graph indicates travel time to protected area. Inclined lines originating from 
the point of the axes’ origin (0,0) represent two types of sUAS approaching the restricted area: 1) 
a typical quadcopter of the DJI Phantom-class drone (with a cross-section approximately 0.25- 
0.3 square meters) traveling at the maximum speed of 45 mph (73 km/h); and 2) a faster fixed- 
wing type drone with the same reflective surface as DJI Phantom travelling at a speed of 100 
mph (160 km/h), the maximum speed for sUAS allowed per FAA regulations. The ranges are 
shown under ideal conditions: flat terrain; no direct obstructions; overcast light conditions 
without precipitation; and typical ambient noise.) Figure 3.1 allows one to make a quick estimate 
of the available reaction time to implement countermeasures after the intruder drone is detected 
by a CUAS system. 


As shown in Figure 3.1, typical radar systems have an advantage over electro-optical and 
acoustic sensors in term of detection and tracking ranges. Note that the typical ranges of RF 
systems are not shown on the graphics. There are two reasons for such exclusion. The first 
reason is that the typical detection range of the RF systems can vary considerably depending on 
the transmitter power output, radio frequency and communication protocol between the sUAS 
and its ground controller. The second reason relates to performance variability related to of the 
types of antenna and amplifier used in different RF CUAS systems. For example, while the 
detection range of the popular DJI portable RF CUAS device equipped with a simple omni- 


directional antenna is similar to a typical control range of DJI Phantom IV drone (about 5 km., or 
3 mi.), the detection range of the stationary system from the same manufacturer with a complex 
array of directional antennae and a high sensitivity amplifier can increase the detection range to 
up to 10 times as far (11). 
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Figure 3.1. Typical detection ranges of various CUAS technologies vs. UAS flight time 


Figure 3.1 also provides a display of challenges associated with the short time to engage CUAS 
mitigation strategies once an intruder UAS has been detected. The problem becomes more 
significant due to the fact that under current regulations only a few federal agencies (the 
Department of Homeland Security, Department of Justice, U.S. Coast Guard, Department of 
Energy, and Department of Defense) are authorized to deploy and implement effective drone 
interdiction technologies (12). The majority of transportation facilities do not have such 
technologies available at the time of a UAS attack and as a result mitigation will most likely be 
limited to less effective passive countermeasures, such as notification of the proper authorities 
and creation of a record of the incident. Another challenge is associated with accurately 
identifying the potential level of threat from a sUAS. For example, in some situations, such as 
near a busy airport, the physical presence of any sUAS can raise a red alert. In other cases, such 
as in the proximity of facilities with large numbers of people and limited emergency evacuation 
abilities, such as bridges, tunnels, or large transit hubs, the presence of an intruder drone may be 
considered a serious threat only when such a drone is carrying an unidentifiable suspicious 
payload. 


Of the 49 CUAS detection and tracking systems identified in Step 1 (see Appendix A, Tables 6.1 
and Table 6.2), 24 systems available on U.S. market were evaluated in Step 2 based on a number 
of performance metrics and other factors (see Appendix A, Table 6.3) From the results of this 
evaluation, 7 systems were recommended for field testing in Step 3 (Table 3.2). More details on 
the systems evaluated in Step 2 are available in the Technical Memo on Literature Synthesis 
produced for that evaluation. 


Table 3.2. sUAS detection, tracking and identification systems recommended for field test 


Detection Detection Range*, Final 
paniraceee oe Technology* km (mi) Score 
Adsys Controls Inc. reek pen eee 10 87.5 

Surveillance Acoustic 
AeroDefence AirWarden RF 7 Po 
DJI Aeroscope (Stationary) RF Up to 50 (30) 75.5 
SRC Gryphon Skylight, Radar, RF, EO, re 87.5 
Gryphon Mobile Skylight | IR : , 
3 (1.8) 
3 Radar, 
Liteye ADIS EO, IR 3-8 (1.8-5) 75.9 
Sensofusion USA Airfence RF Up to 10 (6) 78 
Notes: *Detection range is shown for DJI Phantom-size target, the most common sUAS on U.S. market. 


3.3 Design of the Prototype of the Field Test 


The field test can be briefly described as a “war game” between the individual or a group of 
CUAS manufacturers who will try to protect a designated facility (“defenders”) and an 
individual or a joint group of MassDOT or UMass UAS pilots, equipped with a variety of 
different airframes who will try to access the designated facility (“attackers”). 


The field testing has been designed to allow maximum flexibility to vendors, MassDOT, and the 
UMass research team. This means that the test may be conducted at different times and different 
locations for different vendors or products. 


In order to provide the required flexibility, the field testing design includes six distinct options. 
Those options include the following: 


1. Each selected CUAS product will be tested at a single location; 

2. All selected products that utilize the same technology (e.g. radar) will be tested at a single 
location; 

3. All selected products representing all technologies will be tested at a single location; 

4. Each selected CUAS product will be tested at multiple locations; 
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5. All selected products that utilize the same technology (e.g. radar) will be tested at 
multiple locations; 
6. All selected products representing all technologies will be tested at multiple locations. 


Each design option has distinct benefits and drawbacks that can be presented with a help of five 
major evaluation criteria. The evaluation criteria include the following: 


1. Convenience to vendors, including flexibility of time and location as well as the level of 


exposure of the product to potential competition; 


2. Convenience to the research team, including factors related to level of effort associated 
with organizing, and conducting the event, and data processing; 


Total time needed to conduct the test; 


3. 
4. Total costs associated with conducting the tests; and 
5. Quality and reliability of collected data. 


The major benefits and challenges associated with the six different design options presented in 


Table 3.3. 
Table 3.3. Major benefits and challenges of CUAS field test options 
One One Tech, | All Tech, | One One Tech, All Tech, 
Vendor, One One Vendor, Multiple Multiple 
One Location Location | Multiple Locations Locations 
Location Locations 
Convenience to | Excellent Very Good | Good Fair Poor Very Poor 
Vendor 
Convenience to | Good Very Good | Excellent | Very Poor | Poor Fair 
Researcher 
Total Time Average Low Very Low | Extremely | Very High High 
High 
Total Cost Average Low Very Low | Extremely | Very High High 
High 
Data Poor Fair Good Very Good | Very Good Excellent 
Reliability 


In the proposed field testing, the test site(s) and the facilities to be protected during the tests will 
be selected by MassDOT. The vendor(s) will be granted the opportunity to survey the area 
designated for protection and to install, test, adjust their equipment as needed prior to the actual 
field test. Similarly, the UAS pilots will be granted the opportunity to access the test site prior to 
the test flights to plan the mission and to familiarize themselves with the landscape. 
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3.3.1 Test Site Selection 


In coordination with MassDOT, the UMass research team considered four potential test sites and 
the advantages and challenges of each. 

1. Gillette Stadium at Foxborough, Massachusetts 

2. Joint Base Cape Cod at Bourne, Massachusetts 

3. University of Massachusetts, Amherst, Massachusetts 

4. Fort Devens Reserve Force Training Area at Devens, Massachusetts 


3.3.1.1 Gillette Stadium 

Advantages: Gillette Stadium has the advantage of being a facility with major public events that 
should be considered for protection against non-authorized, non-cooperative UAS. Also, as the 
stadium is a well-known landmark, a test conducted at this facility could increase the vendors’ 
interest in participating in the field test. 


Challenges: The major challenges are related to the requirement to obtain permits to conduct the 
proposed test, potential delays associated with obtaining such permits, and possible limitations 
on the scope of the test due to the nature of the facility. Other challenges may be associated with 
the need to handle the installation of larger and heavier UAS detection products, such as the 
large radar, which should be placed high over the ground in order to achieve optimal 
performance. Finally, compared to the other potential sites, this location probably has the highest 
travel and accommodation costs for the research team and vendors. 


3.3.1.2 Joint Base Cape Cod 

Advantages: The Joint Base Cape Cod has the advantage of being an already established UAS 
test site and a military location that may have fewer restrictions associated with both the 
conducting of UAS flights and on-site testing of CUAS technologies. 


Challenges: The major challenges could be associated with having to obtain permits to allow 
access to the base for certain individuals from both the research team and vendor representatives, 
and the timeframe needed to obtain such permits. 


3.3.1.3 University of Massachusetts Amherst 

Advantages: The University of Massachusetts Amherst campus has the advantage of being a 
home of the UMass research team and the UMTC administrative team. This could help simplify 
the process of getting sUAS flight permits. Also, the campus is familiar ground for the research 
team pilots, who will, therefore, require less time and preparation for the test flights. Finally, the 
process of setting up and conducting the experiment as well as costs associated with travel and 
accommodations on campus during the testing are expected to be the lowest if this test site is 
selected. 


Challenges: Due to the large number of people working and/or living on the UMass Amherst 


campus, more time and effort for planning, scheduling, and safety considerations may be 
required. 
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3.3.1.4 Devens Reserve Forces Training Area (RF TA) 

Advantages: The Devens RFTA has the advantage of being an already established UAS test 
location which may eventually become a part of a large proposed UAS test corridor between 
Massachusetts and New York. It is also a long-time military installation that may have fewer 
restrictions associated with both the conducting of sUAS flights and on-site testing of CUAS 
technologies. 


Challenges: Major challenges could be associated with obtaining permits to get access to the 
base for certain individuals from both the research team and vendor representatives, and the 
timeframe needed to obtain such permits. 


3.3.2 Additional Details for Consideration 


It is desirable that the design of the field tests include both common challenges and technology- 
specific challenges to evaluate CUAS products. Common challenges include ones to help 
evaluate the ability of CUAS to detect and track multiple drones representing various platforms 
approaching the protected facility at different directions, speed, and altitude. Technology- 
specific challenges access and evaluate each product’s vulnerabilities as described in the 
literature synthesis. Such technology-specific challenges may include: a low-altitude terrain- 
following approach to test radar capabilities; an autonomous flight in a near radio-silent mode to 
test RF-intelligence CUAS systems; an approach during the poor visibility to test EO systems; 
and an approach conducted in a noisy environment to test acoustic sensors. It is also desirable to 
conduct a test of multiple technologies working together under a combination of unfavorable 
conditions. Table 3.4 provides a brief summary of the strengths and limitations of CUAS 
technologies, as evaluated by the researchers based on performance criteria. The summary could 
serve as a guide for designing various CUAS challenges. 


Table 3.4. CUAS technology performance indicators 


Counter-UAS Technology Solution 

Performance Indicators Radar RF Electro-Optical Acoustic 
Intelligence 

Range Excellent Excellent Fair Poor 
Target Tracking Good Good Good Fair 
Target Identification Fair Excellent Excellent Good 
Payload Identification None Fair Excellent None 
Low Light Excellent Excellent Fair Excellent 
Urban Landscape Fair Good Fair Good 
Noisy Environment Excellent Excellent Excellent Poor 
Weather Precipitation Excellent Excellent Poor Fair 
Rogue Drone Excellent None Excellent Excellent 
Ability to Locate Pilot None Excellent Fair Poor 
Difficulty of Installation High Medium Low Low 
Difficulty of Maintenance Low Low High Average 
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3.4 Field Testing and Demonstrations 


As was discussed earlier, due to a number of logistical constraints, rigorous field testing of 
CUAS technologies was unable to be conducted during this study. The constraints against field 
testing included the regulatory restrictions (see Appendix D for details), lack of time to finalize 
the format of the field tests due to project evolution based on MassDOT priority needs, the 
challenges of selecting the best format and location for the testing, and of coordinating with 
multiple vendors, and limited funding for conducting the tests. (only a few thousand dollars in 
total were allocated for field testing in the project budget). 


In lieu of field testing, there were a number of field demonstrations conducted by individual 
CUAS vendors and attended by MassDOT staff and others. Those demonstrations did not 
include rigorous testing of CUAS products under a variety of conditions, but the results of the 
demonstrations were still informative. 


One such CUAS demonstration was conducted in June 2019, in Foxborough, Massachusetts near 
Gillette Stadium, and attended by MassDOT staff, UMass research team members, as well as by 
various public officials and researchers. The demo, conducted by an invited CUAS vendor, 
included an “invasive” drone and a “defender” CUAS drone. The demonstration was conducted 
in a closed parking lot, and a safety perimeter was established around the demonstration area. 
The demonstration provided effective detection, identification, tracking and capture of the non- 
cooperative “invasive” drone flying in a “silent” autonomous mode without radio 
communication. At the beginning of the demo, the vendor’s personnel described the CUAS 
technology and the demo procedures. During the demonstration, the CUAS “defender” rapidly 
detected and tracked the invasive drone using radar. After tracking the invasive drone, the CUAS 
drone sent a message to the ground operator asking for permission to launch the CUAS drone. 
During the demo flight, both the defender and invasive drones operated autonomously using 
GPS, with pilots standing by ready to intervene in case of emergency. As the defender drone 
approached within striking distance of the invasive drone, it asked the ground operator for 
permission to capture. Once permission was granted, the defender drone quickly captured the 
invasive drone. Finally, the defender drone brought the captured invasive drone back to the 
launch area, so that the invasive drone and the disabling technology on board the CUAS drone 
could be examined. 


MassDOT may consider incorporating rigorous field testing into a future round of CUAS 
research. 
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4.0 Conclusions and Recommendations 


Designed to expand and update the results of the Phase I study on CUAS technologies for 
protecting airports, the Phase II study was conducted to review technologies available to detect, 
track, and identify sUAS near critical ground transportation infrastructure, including that located 
near or within densely populated metropolitan areas. 


Consistent with the findings from Phase I, as well as similar reviews and field tests conducted by 
others (13, 14, 15, 16), this study have found that there is no CUAS product that utilizes any 
single type of sensing technology while at the same time being capable to address all challenges 
associated with sUAS detection, tracking, and identification. The most promising technologies 
include RF signal intelligence, EO systems, acoustic signature techniques, and surveillance 
radar. Each technology has distinctive advantages and drawbacks related to its capabilities, 
reliability, and capital and operating costs. Hence, the research team recommends to select 
products that combine multiple UAS detection, identification and tracking technologies that 
would provide the most robust protection for critical transportation facilities. 


Based on a preliminary evaluation carried out by the research team, seven commercially 
available CUAS products have been identified for field testing. The evaluation of selected CUAS 
products was based on parameters and capabilities provided by the manufacturers. The 
evaluation parameters included: detection, tracking, and identification ranges; the ability to 
detect and identify payload, operate in adverse conditions, and detect rogue drones that operate 
in a fully-automated, radio-silent mode; and some non-technical parameters such as capital and 
operational costs, regulatory compliance, potential collateral damage and environmental impacts. 


The prototype of the field test was designed to evaluate CUAS products that represent the most 
promising CUAS technologies for detecting, tracking, and identifying cooperative and non- 
cooperative SUAS. The field test would assess the selected CUAS products for their 
performance, capabilities, and reliability for protecting critical surface transportation 
infrastructure. It is recommended that a prototype for the field test be conducted upon the final 
approval of the testing design and location(s) by MassDOT during the next phase of UAS 
research. 


It is expected that the results of this study will be of interest to a variety stakeholders including 
State DOT officials; FHWA, FAA, the Federal Transit Administration (FTA), the Federal 
Railroad Administration (FRA), and other federal agencies; transportation security and law 
enforcement agencies; university researchers; transportation facility operators, contractors, and 
consultants. 
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6.0 Appendices 


Appendix A: UAS Detection and Tracking 


Systems 
Table 6.1. UAS detection and tracking systems: U.S. manufacturers 
Type(s) of wey 
Manufacturer Product Name yP Detection Range, km (mi) Page 
Sensor(s)* F 
Link 
Adsys Controls Inc. SATS2 Aerial Surveillance |EO/LiDar, Acoustic Up to 10 (6) 1 
AeroDefence AirWarden RF Up to 7 (4.4) 2 
C Speed LLC LightWave Radar Radar Up to 10 (6) 3 
RE-100 RE 2 (1.3) 
Dedrone 4 
RE-300 RE 1.5 (1) 
DroneWatcherRF Mini RF 0.5—0.8 (0.3—0.5) 
DeTect DroneWatcherRF RF 3.2+ (2+) Pp) 
DroneWatcher DSR Radar 3.2+ (2+) 
DD610AR Stationary Drone RE 1 (0.6) 
Drone Labs petedtor 7 
DM610R Portable Drone iz 
RE 1 (0.6) 
Detector 
DroneShield DroneSentinel Radar, RF, EO, IR 5 (3) 8 
Dynetics GA 9000 Radar 5 (3) 9 
eae Gryphon Skylight Radar, RF, EO, IR 10 (6) ia 
Gryphon Mobile Skylight |Radar, RF, EO, IR 10 (6) va 
Liteye ADIS Radar, EO, IR 8 (5) 11 
Sensofusion USA Airfence RF 10 (6) 12 
A150 A-Series Counter- 
Drone Radag Radar 0.2 (0.13) 
A600 A-Series Counter- maaae 0.6 (0.4) 
Drone Radar 
SpotterRF : 13 
A3000 A-Series Counter- Baas 0.7 (0.5) 
Drone Radar ee 
A2000 A-Series Counter- 
Drone Radar Bade (G8) 
TCI Blackbird RF N/D 14 
UMass/Raytheon CASA Radar Radar 1 (0.6) 15 
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Table 5.2. UAS detection and tracking systems: Manufacturers outside U.S. 


Detection Range Country of mek 
Manufacturer Product Name Type(s) of Sensor(s) Be, wank Page 
Origin ‘ 
Link 
Exponent DroneHunter EO, IR N/D U.ALE. 1 
Dil AeroScope (Stationary) |RF Up to 50 (30) China 2 
Aeroscope (Portable) RF Up to 5 (3) China 2 
Groupe AD EDEN A: eoiaaiia Radar, RF, EO 5 (3) France 3 
Services 
HGH Infrared 
Systems Spynel M IR/EO 1.5 (1) France 4 
Kelvin Hughes SharpEye Radar N/D U.K. 5 
Squarehead Discovair Acoustic 0.5 (0.3) Norway 6 
ADS-2000 Acoustic N/D 
_ SC-1000T EO, IR N/D . 
Meritis Switzerland 7 
SC-1500T EO, IR N/D 
SR-9000S Radar N/D 
Microflown AVISA SKYSENTRY Acoustic 0.4 (0.25) Netherlands 8 
Miltronix Drone Detection Radar |Radar 4 (2.5) U.K. 9 
EAGLE Radar 1 (0.6) 
WINGMAN 100 RF 1 (0.6) 
Mydefence Denmark 10 
WATCHDOG RF 1 (0.6) 
WOLFPACK RF 1 (0.6) 
NEC EO, IR, RF, Acoustic 1 (0.6) Japan plu 
TUNGSTEN Radar Up to 12 (7.5) 
a V3 Radar Radar 1.6 (1) 
Quantum Aviation U.K. 2 
TITANIUM RF N/D 
CHROMIUM EO, IR 4 (2.5) 
Rinicom Sky Patriot EO 0.8 (0.5) U.K. 13 
Robin Radar Systems {Elvira Radar 3 (2) Netherlands 14 
. : ; SkyDroner 1000 EO, Other 0.5 (0.3) : 
TeleRadio Engineering Singapore 15 
SkyDroner 500 EO, Other 1 (0.6) 
TRD Consultancy Orion-D RF 4 (2.5) Singapore 16 
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Table 6.3. CUAS detection, tracking and identification product evaluation 


Primary Secondary Non- 
Product Performance Performance Performance 
Parameters Parameters Parameters 
< (3) Pr PM a 
=| on 3 =| & | x = = = a 
| =| o io] i) a ro) a 0 
is aS Y = 4 = Q ° 77) I =| 
Manufacturer Model o|vl| & o|i83inzulolxe!s| als 
S/E/ E/Sl sls] a) si Cl] Els 
AO} & a Py s > eo ° z 
= fa ou OC a 
Adsys Controls Inc.* | SATS2* 10 | 10 10 H A | A | H A P H H 
AeroDefence* AirWarden* 7 7 7 A H H P H A H H 
C Speed LLC LightWave Radar 10 7 3 P A H H P P H P 
ecieae RF-100 2|7 7 |A/lTH/IH{IP|J|HIJHI|A ISH 
RF-300 15 7 7 A H H P H H A H 
DroneWatcherRF 
DeTect Mini 0.8 7 7 A H H P H A A H 
DroneWatcherRF 3.2 A H H P H A A H 
DroneWatcher DSR_ | 3.2 P A H H P P H A 
AeroScope 
DII* (Stationary)* 10 7 7 A H H P H A A H 
Aeroscope 
(Portable) 5 vi i A H H P H H A H 
Tine abe DD610AR i ee 7 |A/lTH/IH|P|J|HIJHI|A lH 
DM610R 1 7 7 A lal H P H H A H 
DroneShield DroneSentinel 5 7 7 H A H H H P A P 
Dynetics GA 9000 5 7 3 P H H H P A H P 
aes Gryphon Skylight* | 10] 10 | 10 |H |H |H JH |H ||P |A JP 
Mobile Skylight * 10 10 10 H H H H H P A P 
Liteye* ADIS* 8 8 8 H A H H H P H P 
Sensofusion USA* Airfence* 10 7 a A H H P H H A H 
SpotterRF A150 0.2 7 3 P A H H P A H A 
A600 0.6 7 3 P A H H P A H A 
A3000 0.7 Vi 3 P A H H P A H A 
A2000 7 3 P A H H P A H A 
TCI Blackbird vi 7 7 A H H P H H A H 
UMass/Raytheon CASA Radar 1 7 3 P A H H Pp P H P 


Notes: These ratings were assigned by the research team. The primary performance parameters were rated on a 
scale 1 to 10 with 10 being the best. (“Detection” column displays product detection range in miles as reported by 
the manufacturer.) The secondary performance parameters and non-performance parameters are rated as poor (P), 
average (A) or high (H). CUAS products recommended for further evaluation in the prototype of the field tests are 
marked with an asterisk in the first two columns (Manufacturer, Model). 
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Appendix B: UAS Interception and Interdiction 
Systems 


Currently, several methods have been considered to deter, immobilize, or destroy invasive 
drones in flight-restricted areas. Such methods include the following (17): 
e RF or GPS signal jamming (can be either wide-area or targeted) 
GPS spoofing 
RF hacking 
Flight disruption by use of electromagnetic or laser impulse 
Flight disruption by use of kinetic means (either destructive or non-destructive) 


The first method, RF or GPS signal jamming, utilizes two different approaches. The first 
approach uses a broad-spectrum, wide-area signal jamming. The second approach uses narrow- 
beam/narrow-RF spectrum antennae to disrupt a drone’s operation and bring it down to the 
ground. 


The second method, GPS spoofing, deceives a GPS receiver by broadcasting a signal with 
incorrect GPS coordinates and forces a UAS to change its initial path and landing point. 


The third method, UAS RF or communication link hacking, hijacks an operator’s control over 
their drone and, for example, sends a command for immediate landing as the UAS enters a 
restricted area. 


The fourth method disrupts a UAS flight using electromagnetic or laser impulse by either 
damaging the electronic components on the UAS circuit board or the drone itself to bring down 
invasive UAS. Finally, the fifth method, flight disruption by physical means, implements 
physical objects to bring down invasive UAS, either without destruction (such as Drone 
SkyWall, a net and parachute combination) or destructive (such as guns or other weapons). 


There are also non-technology-based methods—such as the use of predator birds—which, 
though considered exotic, have proven to work well on small- to medium-sized drones (16). 
Such solutions are beyond the scope of this research due to the lack of testing or reliability 
records in the United States. 


6.4 provides a brief summary of the advantages and drawbacks of various drone deterrence and 
interception methods. 
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Table 6.4. UAS deterrence and interception methods: Pros and cons 


Kinetic 
RF Signal GPS RF El-mag or : 
Jamming Spoofing Hacking Laser Destructive | Non- 
Destructive 
Advantages | Wide area of | Wide area of | Low Effective Effective Effective 
coverage; not | coverage; not | interference | against all | against all against all 
labor- labor- and UAS UAS UAS 
intensive intensive collateral 
damage 
Drawbacks | Potential Potential Ineffective Collateral | Collateral Short range; 
interference; | interference; | against fully | damage; damage; labor 
ineffective ineffective auto UAS legal legal intensive 
against fully | against fully limitations | limitations 
auto UAS auto UAS 


The research team identified 54 interception and interdiction system available either in the U.S. 

or internationally. Brief details of selected UAS interception and interdiction systems, embedded 
technology, and their major technical parameters are presented in Table 6.5 (U.S. manufacturers) 
and Table 6.6 (non-U.S. manufacturers). 


Table 6.5. UAS interdiction systems: U.S. manufacturers 


inierdicion Interception |Web 
Manufacturer Product Name Range, km Page 
Method(s)* : ‘ 
(mi) Link 
Battelle Drone Defender V2 C-UAS RF/GNSS jamming 0.4 (0.25) 1 
CACI Small Form Factor RF jamming N/D 2 
Dedrone RF and GPS Jammer RF/GNSS jamming N/D 3 
DroneGun MKII RF/GNSS jamming 2 (1.3) 
DroneShield 4 
DroneGun Tactical RF/GNSS jamming 1 (0.6) 
IXI Technology Drone Killer RF/GNSS jamming 0.8 (0.5) 5 
NASA 
Langley Research Center Safeguard System Net capture 0.4 (0.25) 6 
Dronebuster Block 3 RF/GNSS jamming N/D 
Radio Hill Technologies 7. 
Dronebuster FS REF/GNSS jamming N/D 
Repulse 24 RF jamming 1 (0.6) 
Repulse 2458E RF jamming 1 (0.6) 
Repulse 
Repulse 2458H Handheld RF jamming 1 (0.6) 
Repulse 360 RF jamming 2 (1.3) 
SCI Technology AeroGuard Net capture N/D 9 
Sierra Nevada Corporation SkyCAP RF jamming N/D 10 
Theiss UAV Solutions ree etic Net capture N/D 11 
Netting System 
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Table 6.6. UAS interdiction systems: Manufacturers outside U.S. 


Interdiction Interception Country of wee 
Manufacturer Product Name P : ee Page 
Method(s) Range, km (mi) |Origin Link 
CTS Drone Jammer RF/GNSS jamming _|No data China 1 
Delft Dynamics DroneCatcher Net capture No data Netherlands 2 
JAM-1000 RF/GNSS jamming [0.3 (0.2) : 
Digitech InfoTech JAM-2000 RF/GNSS jamming _|1.2—2.1 (0.7-1.5) |“"™ 3 
JAM-3000 RF/GNSS jamming __|No data 
Drone Defence Dynopis E1000MP RF/GNSS jamming —__/1 (0.6) UK 
SkyFence RF jamming 0.5 (0.3) _ ie 
Groupe Assman MTX-8 Net capture No data France 5 
Harp Arge Drone Savar RF jamming No data Turkey 6 
HiGH + MiGHTY SKYNET REF/GNSS jamming __|No data Taiwan Z 
Pre Defender Series : . ‘ 
Hikvision UAV-DO4JA RF/GNSS jamming _|No data China 8 
HP 3962 H RF/GNSS jamming __|No data 9 
H.P. M&C ; - Germany 
HP 47 RF/GNSS jamming _|No data 
Jiun An Technology Raysun MD1 RF/GNSS jamming —__/1.1 (0.7) Taiwan 10 
a Recurve RF/GNSS jamming _|No data UK. 
Kirintec 11 
Sky Net Longbow RF/GNSS jamming __|No data 
P6 RF/GNSS jamming _|No data 
RTX-2000M6 RF/GNSS jamming __|No data 
Meritis RTX-3000X RF/GNSS jamming _|No data Switzerland 12 
RTX-300P2 RF/GNSS jamming __|No data 
SkyCleaner RF/GNSS jamming _|No data 
Open Works Eng. Skywall 100 Net capture No data ik ie 
Skywall 300 Net capture No data ie os 
Optix Anti-Drone RF/GNSS jamming /|Up to 2 (1.3) Bulgaria 14 
GROK Jammer RF/GNSS jamming  /|2—4 (1.2-2.5) 
GROK Mobile Gun RF/GNSS jammin: 1 (0.6 
Prime C & T = u - . (0.8) ea 15 
Meritis Jammer RF/GNSS jamming __|No data 
Phantom Jammer RF/GNSS jamming [2 (1.3) 
ae VANQUISH 1 RF jamming No data 
Quantum Aviation U.K. 16 
VANQUISH 3 Net capture No data 
Search Systems Sparrowhawk Net capture No data ULK. 17 
Sentinel Catch Net, parachute 5 (3) . 
Skysec - Switzerland 18 
Sentinel Catch &-Carry |Net, hook 2 (1.3) 
SteelRock Technologies |NightFighter RF/GNSS jamming __|No data U.K. 19 
Ommidrecnonal RF/GNSS jamming —_|No data 
Terra Hexen Jammer Poland 20 
Neutralizer RF/GNSS jamming __|No data 
TRD Consultancy Orion RF/GNSS jamming =//1.5 (1) Singapore 21 
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Appendix C: UAS Detection, Tracking and 
Interdiction Hybrid Systems 


UAS Detection, Tracking, and Interception Systems 


Today, more and more UAS detection and tracking equipment manufacturers are offering a UAS 
interception and control system as a part of their purchase package, as there is a clear demand for 
such all-in-one systems due to increasing awareness about the potential threats presented by non- 
cooperative UAS to important facilities and infrastructure. The research team identified 57 
counter-UAS detection, tracking, identification, interception and interdiction hybrid system 
available either in the U.S. or internationally. Brief details on these CUAS hybrid systems are 
presented in Table 6.7 (U.S. manufacturers) and Table 6.8 (non-U.S. manufacturers). 


Table 6.76. UAS detection, tracking, and interdiction hybrid systems: U.S. manufacturers 


: Web 
Manufacturer Product Name Denon Interdiction Method(s) Page 
Method(s) ; 
Link 
Airspace Systems Airspace EO Net capture 1 
Black Sage/IEC Infrared |UAVX Radar, EO,1R |RF Jamming, ps 
GNSS jamming 
Wireless Intrusion 
Blind Tiger Communication |Detection and Defeat RF GNSS Spoofing 3 
System 
CACI SkyTracker RF RE emune: 4 
y GNSS jamming and spoofing = 
RF jamming, 
CellAntenna D3T RF GNSS jamming 5 
CITADEL DFU3000 RF GNSS Spoofing 6 
Department 13 International |MESMER RF GNSS Spoofing 7 
Heian DroneTracker RF, EO, IR, RF jamming, 8 
Multi-Sensor Acoustic GNSS jamming = 
Radar, RF, RF jammin 
DroneShield DroneSentry Acoustic, Pas ae 9 
EO, IR J 6 
Fortem Drone Hunter Radar Net capture 10 
Liteye/Blighter/ : , 
Chess Dynamics/ECS AUDS Radar, EO, IR _|RF jamming 11 
F RF, EO, : : 
Lockheed Martin ICARUS f RF jamming, 12 
Acoustic 
Orbital ATK T-REX RadawkOnie «| one Fe) 
kinetic 
Rohde & Schwarz ARDRONIS RE RE jammin’ 14 
GNSS jamming 
SESP Drone Defeater EO, IR, RF RF jamming 15 
SRC Silent Archer Radar, FO, 1R |RE Jamming, 16 
GNSS jamming 
Van Cleve & Associates DroneRANGER Radar, IR, EO |RF jamming 17 
Whitefox Dronefox Fortify RF GNSS spoofing 18 
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Table 6.87. UAS detection, tracking, and interdiction hybrid systems: Manufacturers outside U.S. 


Web 

Manufacturer Product Name Detection Method(s) _|Interdiction Method(s) ones of Page 

6 Link 
Airbus Group SE COME AN ‘lreadiaccin RE/GNSS jamming _|France 1 

System 
ArtS YS360 RS500 RF REF/GNSS jamming Israel 2 
Aveillant UWAS Radar, EO, IR RF jamming U.K. 3 
Pence Seeuny Drone Blocker RF RF jamming Netherlands | 4 
Services 
BYLBOS/Roboost SPID EO, IR, RF, Acoustic RF/GNSS jamming France 5 
CerbAir CerbAir RF, EO, IR RF jamming, net capture |France 6 
D-Fend Solutions RF RF jamming, spoofing [Israel vs 
Dronefence RF, Acoustic, EO, IR GNSS Spoofing Germany 8 
Elbit ReDrone RF RF/GNSS jamming Israel 9 
ELTA (sae. _- «| [proneGuard Radar, EO GNSS jamming Israel 10 
Aerospace Industries) 
ELT-Roma ADRIAN RE, Radar, EO, IR; RE/GNSS jamming Italy abl 
Acoustic 
Gradiant Counter UAS RF, EO RF jamming Spain 12 
Hensoldt Xpeller Radar, EO, IR RF/GNSS jamming Germany 13 
IACIT are EO, RE, Acoustic, Radar|RF jamming Brazil 14 
Red Sky 2 Drone : F 

IMI Systems Defender System Radar, EO, IR RF jamming Israel 15 
KB Radar Groza-Z RF RF/GNSS jamming Belarus 16 
L3 Technologies Drone Guardian —_|Radar, EO, IR, RF RF/GNSS jamming U.K. 17 
Mitsubishi Electric Drone Deterrence RE RF jamming Japan 18 
Corporation System 
Derlae ce C-Guard Dronenet |RF RF jamming Israel 19 
Communications 
Orad DROM RF RF jamming Israel 20 
Orelia Drone Detector Acoustic RF jamming France 21 
Phantom Technologies |Eagle108 RF, Radar, EO, IR RF/GNSS jamming Israel 22 
Pomme ConsMiine es a weebrone Radar, IR, EO, Acoustic [RF/GNSS jamming [Denmark | 23 
Technologies 
Rafael Defense oeane Due Radar, EO, IR RF/GNSS jamming, (aeel 24 
Systems laser 
Rohde & Schwarz ARDRONIS RF RF/GNSS jamming Germany 25 
pneu Guardion Radar, RF, EO, Acoustic |RF/GNSS jammin German 26 
Defence/ESG eater J 6 y = 
Selex Falcon Shield Radar, IR, EO RF jamming U.K. 27 
eieelnock ODIN RE, IR, Radar, EO RE/GNSS jamming —|ULK. 28 
Technologies 
Terra Hexen SAFESKY Radar, EO, Acoustic RF/GNSS jamming Poland 29 
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Appendix D: Legal Barriers to CUAS 
Operations 


There are several Federally-mandated legal barriers to CUAS operations. They include the 
following (18): 


18 U.S. Code § 32: prohibits damaging or destroying an aircraft. 

18 U.S. Code § 1362: prohibits willful or malicious interference with U.S. government 
communications. 

18 U.S. Code § 1367(a): prohibits intentional or malicious interference with satellite 
communications. 

Title 47: requires radio transmitter operators to be licensed or authorized; prohibits 
willful interference with radio communications of any station licensed, authorized, or 
operated by the U.S. government; and prohibits using or generally dealing in (except by 
the U.S. government) any signal “jamming” devices. 

49 U.S. Code § 46502: prohibits “seizing or exercising control of an aircraft...by force, 
violence, threat of force or violence, or any form of intimidation, and with wrongful 
intent.” 

The Computer Fraud and Abuse Act: Creates a long list of crimes prohibiting conduct 
that affects a computer that is “used in or affecting interstate or foreign commerce,” 
including threatening to damage a computer with the intent to extort anything of value; 
“knowingly causing the transmission of a program, information, code, or command, and 
as a result of such conduct, intentionally causing damage without authorization”; 
unauthorized access with intent to defraud or in combination with destroying, damaging, 
or altering information; and trafficking in “any password or similar information.” 

The Wiretap Act: prohibits the use of “any electronic, mechanical, or other device” to 
intentionally intercept, attempt, or have someone else intercept the contents of any 
electronic, wire, or oral communication; disclosing (or attempting to disclose) the 
contents of any such communication obtained by unlawful interception; and intentionally 
using or attempting to use the contents of any such communication. 

The Pen Register Act: prohibits the installation or use, without a court order, of pen 
registers, including any device that “records or decodes” signaling and other information 
transmitted by electronic communication, or a trap and trace device, including any device 
capable of identifying information that reveals the source of an electronic communication 
by capturing an incoming impulse. 


There are also FAA regulations that raise the possibility of additional restrictions of CUAS 
operations. For example, 14 CFR § 107.12 and § 107.19(a) require anyone controlling a drone to 
have a remote pilot certificate with a sUAS rating or to be under the direct supervision of a 
remote pilot in command who has the ability to immediately take direct control of the sUAS. 
This suggests that a CUAS operator might also have to be a licensed UAS pilot. 
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Executive Summary 


This study, Drone Cyber Security: Assurance Methods and Standards, was undertaken as part 
of the Massachusetts Department of Transportation (MassDOT) Research Program. This 
program is funded with Federal Highway Administration (FHWA) State Planning and 
Research (SPR) funds. Through this program, applied research is conducted on topics of 
importance to the Commonwealth of Massachusetts transportation agencies. 


Commercial and recreational Unmanned Aerial Systems (UAS) have gained popularity in a 
wide variety of applications and are anticipated to expand use throughout civilian airspace. 
Potential applications include but are not limited to panoramic photography, three-dimensional 
surveying, transportation infrastructure monitoring, surveillance, and damage inspection, 
search and rescue, agricultural services, and scientific research. UAS are a form of cyber- 
physical system that are composed of both hardware and software elements and are therefore 
susceptible to a variety of attacks that could compromise their security and privacy as well as 
the reliability and safety of individuals and assets in the environments they operate. A risk 
management strategy that addresses how UAS can be integrated in to our national airspace 
must consider these technical risks in regulatory policies and procedures. To properly define 
the impact of cybersecurity on mission risk, it is necessary to assess preflight, inflight, and 
postflight operations. This includes the selection of a UAS and its payload as well as its 
configuration, including the mission profile, conduct of mission, potential data acquisition and 
transmission as well as post processing of data, storage, and reporting. Thus, UAS mission 
security must consider diverse threats such as attacks on hardware that is compromised by 
design, software that is compromised intentionally or due to a poor design, websites for 
mission configuration, mission laptop, wireless communication, and networks and data storage 
facilities. Formal risk models are needed to quantify the nature and severity of consequences, 
so that mitigation strategies can be identified, compared, implemented, and validated. 


Vii 


This page left blank intentionally. 


viii 


Table of Contents 


Technical. Report Doctiment Page ..cesecceteed sisces excandecedvedesestieasi eee nhac wrtwn ee eae i 
PLCKMOWIEG SMES. iu. jecxnet og cahds eiicoceted pea laces dicpeetancancnuilacdiceaieciecesbucaralussnrdeusleocebetieslaleebeareedioearead V 
DDUSCLAIIIET sss 2s so cuis agececeaiy sc eSdt Sos ces sea eapa ve avandia coin ace oobi alana ee aiitedt Vv 
EX CCULIVE SUNIMMALY toscssetivesiecnuckvssis tuk siaceacin ssadtstuvsatacouslonsnal cea dinstadesncsaeaccous ie teavsatiowansmadietveend vii 
SIA LOB GOTT aces strats ect duccR ae emcnec ahaa cotencnseeedict evacia ct oMashenstvbinnts edtcEmeureceacuadamescosns ix 
List OL FaBles. sc. .ists,0sésieSscoratessnnseaeg Suepet pene adaedanpebadonsdded Suoainniaede lnsbandepaiuwosdeeddarommeriesuebaneebebeeens xi 
MSISUGE BUG LOS ves rl. accnensecaneddveete tance Card ak ceoo oaeaisees tere ee aUl adkoesste ocean eiesete an ecaad auras xiii 
Mist Of ACrON VUS, Sccechsy odes sceveceetdes sce iajuieceataabedcsaipuscastiessiteus aistenchat cdast weeag dae a enemas desea a 
CSO TOGO oc toe nes eal otese sete ie steapa a tca tv suede ae esa re ccgar ua SUsNaate sacs ie ctce oe asi stuee Teena 1 
PT SCOPC OP StU ys secceieceeilecdekaeecd ives vie cavce extwsvad en cduee chs sae Wdes send cvvecote cots Saude weaavetes detwacebeaevescesdbeusenenaeels 1 
Ve2- Finding vccssccs seevvecdeeszas chdeedvb dees heseglatadecteesnac dani nsateeedeccudevanccvansunc chda cubccdhsaeccesecuvsesentaus cideateccdensastedes 2 
did RBGOMUMENCAUIONS 33455 ra ctuutecesecaibenmneesivaeedbsrsecuatauvs ekaiee weueeciuesbebnsactua teas ntavea una esius tases ius uebie pines 2 
2.0 Representative MassDOT UAS Mission and Cybersecurity Risk...........cesessscesseeeteeeeeeeees 5 
2.1 Technical Decomposition of UAS Mission for Risks Identification ............. eee eeeeeceseceeeeeceeeeeeees 6 
3.0 Risk Assessment and Mitigation................:cccscccsscscerssssscssccesscessssesscessccesscesassenscessccesseesnnsenss 9 
3.1 Cyber Risk Enumeration Example ............c:cccsscsssssssecscecsseceseceseceseceseseceeececeneceneeeseecsaecaaecsaeceaeceseees 10 
Bel PIRISk Cate Gores esevesciieks vsesvacieaccteeseees ch aavceissedheaeirewctieaavostceuls citaaaseetsluivascilinene vescdacioeeaeebes 11 
3.1,2-Risk Assessment Example vy: ciscccecevesveriecivessstaaveccedsactsdsti es veddviedeaoenieienteantecaveetiessexedvercvseess 11 
3.1.3 Cyber Risk Stoplight Charts ..........cccsesssccssecscecsseceseceseceseceeceeseseseseneeeneeensecsaecaaecaeceseeeseees 13 
3.2 Countermeasure Portfolio Selection Problem 0.0.0... .eescsssscesecsseeeceeecesecseeencesecseeeaeeeesaeeaeeeneeaes 13 
3.3 Countermeasure Selection []UStration ...........ceeecsseeseeeecesecseeeecesecsaeeaceeceseceeeeneesecseeeeeeeeaeeaeeeeeaes 15 
3.3.1 Cost of Implementing CounterMeasures............eeesccessesceeeceseceeeccesecaceeceeeeaecaeeeeeeseeseeeeees 16 
ASA ATS va scicss oi2flediccntlsinnidenySiadeandsinatedilodanseadusonscabe bangian tana bates iaasinubeswasd suet aedounacwanedin baapeitepeans 19 
AAA erial Systems. Sater y cise te sisedecsssdecsderesestasscesdec vbobinesssvaatesTeeestedavtedasostecades osidediesiecuabesstecdesVaciueasudvets 19 
4.1.1 Safety: Department of Defense Standard Practice System Safety 882E............eseeeeeeees 19 
4.1.2 Airborne Systems: DO-178C Software Considerations in Airborne Systems and 
Equipment Certification: .s.. 2: ssenevniiieiea a nievissh nate eicanmea et aedannitindan ans 20 
4.2 UAS Navigation and COMMUNICATION. ............ccceesseesseeseeeeseesceessecseeceaecaeceseceseceseeseseseneseneseneeenaeenees 20 
4.2.1 Navigation: RTCA/DO236B Minimum Aviation System Performance Standard............. 20 
4.2.2 Communication: IEEE 1609 - Family of Standards for Wireless Access in Vehicular 
ENVIrOnmMents:CW AVE tose: waecssececesceteetsccivazsescte BecceueieaesseleelecestocecssQiabenceie stevatesveseussbanteteeereeveet sae 21 
4.3 Cyber Test and Evaluation and Risk Management ..............ssssssscssecseeeecesecseeeeceecsaeeaeeeeeaecaeeeneeaes 21 
4.3.1 Cyber Test & Evaluation: Department of Defense Cyber Test and Evaluation Guidebook 
M.CFSION 2:0. oiccs etieestgsteewense Shes les sicosssneduundue cedvstys cheesbpdss svedhl oaveeteusedvehan cols bunetedew restos bngeeddeskneateentesies 21 
4.3.2 Cyber Risk Management: National Institute of Standards and Technology Risk 
Management Framework (RME)..........:ccsscesscsssceseeeceeesseeseeessecseecsaecseceseceseceseeseneseeeeeneeeneeeneeenees 22 
SiO RELCRON COS 25: wieatecseagesas ida ciecutaire dos lusaepa beatae landtadetcbasiien Mestad cxehar aba lauetadebasatinn Sieescersgecune bees 23 
UT MCLE as sactens 5 cash s3 Sones pacanes Nose aed nasheed cc canacaauteas tAeaadunae Sotadaacomnaeananaea atone 26 
Appendix A: Additional Risk Evaluation Metrics 0.0.0... cscsesescesssereessesecseeeecesecaeeacesesaecaeeeneesecseeenees 26 
Appendix Be RiSkS::.ccsitssecastaccceusvovteacitoakd eats ods cet awcceusvedutesreceteaneesatdeteceataa beet ediiease guest eavdaciieaseabes 28 
Appendix: GA ttacks trite veccesteeninteaveadeeete i anievss tener enieninieeaennete er eden nites 37 
Appendix D: CounterMmeaSures............ccsccesscesssssscssseecseceneeesscessecseecsaecesecesecsecesecsssesceseseseneseneeeneeenaeenees 39 
Appendix E: Functional Decomposition and Data-flow .0....... ee eceeeeccesecseeeeceseceeeeceeeeaecaeeeneesecneeeeees 42 
Appendix F: Attacks Categorized According to Mission Stage and Category ...........cssescseeeeeeeeceeeees 50 


ix 


Table 3.1: 
Table 3.2: 
Table 3.3: 
Table 3.4: 
Table 3.5: 
Table 3.6: 
Table 3.7: 
Table 3.8: 
Table A.1: 
Table B.1: 
Table B.2: 
Table B.3: 
Table B.4: 
Table B.5: 
Table B.6: 
Table C.1: 
Table D.1: 
Table D.2: 
Table E.1: 
Table E.2: 
Table E.3: 
Table E.4: 
Table F.1: 
Table F.2: 
Table F.3: 
Table F.4: 
Table F.5: 
Table F.6: 


List of Tables 


Risk evaluation metrics for UASS ...........cssscssessesrcesscosstssccsoteatscocerscoaccnacesereanecnsenss 9 
RSD A IAI see as dt cates dl ac ev nsaaa ead Pact tas catia th caamia feet Pouucoher ys aes au 11 
“Ely away Cy DEL LISK CVANIALION :cscas ce svsacscasaasdsveasescacetaacascaaeecsvavsaniaetaausevecseticasraass 12 
Cyber risk assessment 1tatrix > sc. cteyexciesccsdevsconen ded ole tesdusntacinsddvdewdeosewce dd thoes Recs ads 13 
Interpretation of cyber risk assessMeMt Matrix ..........cesceseesceeeeseeeeeceaeceeeeaeeeeeeeeees 13 
Various measures for attack and COUNtEFMEASUTE .......... eee eeeeeseceseeneeeneeeeeeteeeaeeneee 16 
Effective risk to cost ratio for countermeasure iMPAaCt.............:cceeeceeceeesceesteeeeeeeees 16 
Iterations of greedy algorithm for countermeasure Selection .............:csseeseeeeeeeees Ly 
Graphical cyber risk assessment template .............c::cceseceseceeseeeseeceeeceeeceeeeeeseeeaeees 26 
BES WAS sieh soda a aecnecdue agate aitcad eae areata tele tvas edt ccceraaieoua ea 30 
MOSS Ol GPS acca creas ctasooteautu seas cantatas pans oddata ceaha cea atndcagnng sau eather anaes 31 
TsSS: OF Datei Tak yotecy sossenseavea neds eohasete heekac Gack sacs ecw naaceseree co edie eis 32 
CLASIT: ojcknvshsyuacasevsacontiyysaduveyvacsateataadussaveceuseessidsvssanian levees danesgadedenas suseva tyensalevbaadeaevenws 34 
Autopilot Software Error/Fail ...............scccsscssssssssccssccesscesessssacsesscessceensssacessseesnees 35 
OCS BAL daca oc ies desea pces oem te cn teudl ster Ora Saise hunt bh eau B bears heen wicca 36 
List of attack mechani sins y:.3:3,)cessessicuiceicdundosnipnasadevdavneatisaebdsaspeniisendyantederduaguwiagen' 37 
COUMECRMICASTITCS s gecctaserdes deaicat casaSccitetveonnarhevdce tahccene di ceacsteseeceseostea dicots deca stanianes 39 
Mitigation effectiy eness Ott OTS as saacs.dc2cvend eaececetess obsneetedexata teas dguntgasivierssd ates 39 
Functional modules and potential attacks on Navigation..............scsscseeseereeeeees 43 
Functional modules and potential attacks on data Collection............sceeeseeseeeeees 45 
Functional modules and potential attacks on Communication ..............seseeeeeeeees 47 
BUD TC COMING Fe ioccocmrantesatersancicaoetacasoaiavearonnseassenen + uceesitnddiveuencantarens evan seastueanuent 49 
PHOT BNE SOT Wale AULACKS sreniaevy cy enessauesnctseap eaeiouive nc sgass exeseecvbuedos aneteie eeseduanednoe oH 
Preflight Nardware attacks vjucississsssuseisaveaciessasess soundetevdadiatvetosalevssisussacassasdastalontesns 52 
ImPliBht: SOL CWare ALLACIES:. coiasssicesoizetoseesntecesacta ia sensteasiauietaseeenrudtatienaae acai 53 
WTB Ht Hardware Atta CiS gfe paccusyavai leave Acta tess ieedeoeiyeadinsa wen aiatd eo onkyaeteen 54 
Inflight Communications attackS............cseeeessessecsseceeeeeceeceeseessecsaeceeeeaeeeneeseeeaeenaes 55 
Inflight: physical Security AttaCKS :c.yocho5 eusacecaneedesveregncvadeaxeiseuetacd siepelaioutoneaavensred She, 


Xi 


This page left blank intentionally. 


xu 


List of Figures 


Figure 2.1: An example of MasSDOT Mission .................ccsecesceseeeoeseeceesceseesecescesoesencensceseeseeeeees S) 
Figure 2.2: Technical decomposition UAS mission for Risk Identification............ 0. eee 7 
Bigure:o. Attack risk model: OF WAS. tetsvsaecainse ihe varade vec duyiaiadedbstiosthedavadde tebe reneseioseleneys 9 
Figure 3.2: Graphical cyber risk evaluation of “Fly AWay” ...........scesseseeneeeeeeseceaecneeeneeeneeeees 10 
Figure 3.3: Graphical representation of countermeasure portfolio selection..............:eseeeeee 15 
Figure 3.4 Fly away risk attack countermeasure dependencies ............:cesceecceseceeeceeeereeeneeeees 15 
Figure3:5) Risk reduction: Pareto: 110 1b isis aaiscasaecs vesesedasaacssieaes sovdezansatcaavedeaavszovasqasvauseasticereaaee 17 
Figure A.1: Graphical cyber risk assessment template ..............sceseeseeseeeceeseeeeeceseceeeneeeneeeeees 27 
Figure E.1: UAS Functional Modules and Data-flow ......... ec eeseseesseceneeeceeeeeeeceaeeeeeeneeeneeeeees 42 
Figure E.2: Functional modules and potential attacks on data Collection.............ceeeeereeeeees 44 
Figure-E. 3% C omimilimeation moat orc. cccezentnis ticyececerend cceatcratd aia inte nicaiartid boaters iaeust eae 46 
Figure E.4: Functional modules and potential attacks on flight control... eee eeeeeeeeeees 48 


xiii 


This page left blank intentionally. 


Xiv 


List of Acronyms 


Acronym Expansion 

ANSI American National standard Institute 

CAPEC Common Attack Pattern Enumeration and Classification 
COTS Commercial-off-the-shelf 

FAA Federal Aviation Administration 

FHWA Federal Highway Administration 

MassDOT Massachusetts Department of Transportation 

SPR State Planning and Research 

UAS Unmanned aerial systems 

UAV Unmanned aerial vehicle 


XV 


This page left blank intentionally. 


XV1 


1.0 Introduction 


Commercial and scientific entities are aggressively exploring a variety of Unmanned Aerial 
System (UAS) applications that would occupy our national airspace, requiring government 
provide regulatory guidance to protect public and private property as well as to ensure the 
safety and privacy of individuals. As a form of cyber-physical system, UAS and their 
supporting computational infrastructure are susceptible to cyberattacks on their hardware, 
software, communications, and data. Technical gaps and uncertainty have a cascading effect 
on the clarity and completeness of policy. Cyber risk management can identify threats and 
quantify their potential impact in the context of an organizations mission and business 
processes in order to systematically allocate limited resources to reduce the probability and 
consequences of cyberattacks. In the absence of comprehensive standards, such high-level risk 
assessment and proactive mitigation planning can inform technology evaluation practices for 
buy, build, configuration, and maintenance decisions as well as routine test and evaluation 
procedures intended to inspire confidence in the security of a system or process. Quantitative 
risk assessment can also support budget justifications for additional work where remediation 
is most needed. 


1.1 Scope of Study 


This study focused on technical risks to UAS missions that may be performed by MassDOT 
or its contractors, but is also relevant to UAS operating within MA airspace and can therefore 
inform broader regulatory discussion on cyber risk management. A MassDOT UAS mission 
was attended by UMass researchers to better understand the problem context and best serve 
MassDOT needs. Primary technical risks considered include UAS hardware, software, and 
communication as they contribute to functional capabilities employed during missions. 
Functional decomposition was conducted to identify common attacks and paths within primary 
UAS modules, including navigation, data collection, communication, and flight control. 
Moreover, attacks were categorized according to mission stage, including preflight, inflight, 
and postflight and mapped to the MITRE Common Attack Pattern Enumeration and 
Classification (CAPEC), a comprehensive dictionary of known patterns of attack employed by 
adversaries to exploit known weaknesses in cyber-enabled capabilities. Relevant standards 
were reviewed, including aerial systems safety, navigation and communication, and cyber test 
and evaluation/risk management. Literature surveyed concentrated on UAS testing, risk 
modeling, and UAS architectures. A stop light chart method for cyber risk assessment was 
adapted from the safety domain. The quantitative risk management framework considers 
attacks, their likelihood and impact, and alternative deterrent and defensive countermeasures. 
To compare alternative mitigation strategies, a countermeasure allocation problem has been 
formulated. A high-level discussion places selected UAS commercial-off-the-shelf (COTS) 
technologies employed by MassDOT in the context of the proposed quantitative risk 
management framework. 


1.2 Findings 


Risks can be introduced at every stage of the mission and business process. Inflight risks are 
commonly the focus of attention, due to safety concerns, but pre and post flight risks are 
equally if not more important. Preflight risks include the acquisition, assembly and 
configuration of the UAS hardware and software as well as multiple web-based applications 
that pose both security and privacy threats. Post flight risks include data processing and related 
storage infrastructure that threaten privacy. The business process helps define the mission 
process. Therefore, business processes can serve as a gatekeeper to mitigate technical risk 
before it is introduced. Standards are necessary but not sufficient. Specifically, domain specific 
standards often fail to recognize the shift toward software-enabled capabilities or prominently 
emphasize corresponding cybersecurity risks introduced by implementing such functionality 
in software. As a result, these standards regularly fall short of offering references to 
quantitative procedures that can enable desired decision support capabilities such as design for 
security and cyber risk mitigation. We identified the need for simple quantitative procedures 
to assess cyber risk, compare the effectiveness of alternative countermeasures, and 
communicate related findings graphically to MassDOT who must also consider the broader 
business context. 


1.3 Recommendations 


The primary recommendations are to (1) survey the MassDOT UAS mission portfolio to 
identify where cyber risk assessment can be applied for the greatest benefit and (2) assess and 
certify humans and UAS to prevent and close gaps in the mission and business processes of 
the organization/agency: 


e Assess the MassDOT UAS mission portfolio. Risk mitigation must focus limited 
resources where they will be needed most. No process or system is entirely secure and 
making oneself a less attractive target is an effective first step toward protection. 

oO Survey present and future trends in the types and frequency of UAS missions to be 
carried out by MassDOT and its contractors in order to identify gaps and prioritize 
cyber risk modeling and mitigation efforts. Concentrate process, elaborating on 
dimensions where business risks are greatest and the volume of missions is the 
highest. 


e Assess and certify humans and UAS systems. Cybersecurity is both a social and 
technical problem. 
oO For the human dimension: 
= Document best practices in pre, during, and post flight mission operations 
and develop lightweight training and certification procedures for employees 
and contractors to ensure best practices are followed and updated 
periodically. 
= Consider technologies that enforce good practices as part of the technology 
assessment process. 


oO For the system dimension: 

= When possible assess, extend, and adapt existing IT security procedures to 
UAS and revise existing IT security policies and procedures that involve 
UAS to ensure consistency and simplicity. 

= Specify standard mission payloads. Consider using only the technology 
needed to complete a mission in order to avoid introducing unnecessary 
risk. 

« Identify approved/disapproved lists of hardware, software, and services to 
streamline the UAS certification process. 

= Conduct cyber risk assessment and mitigation studies based on the 
MassDOT mission portfolio. Specify single mission platforms where 
feasible to avoid concentrating risk that would require a more costly and 
complex portfolio of countermeasures to protect a multi-mission UAS. 


Ensure standards reference cybersecurity clearly before endorsing. MassDOT staff 
should consider participating in the working groups of Standards through their 
affiliated experts to ensure that cyber risk concerns for Massachusetts specific to 
transportation are adequately represented. 
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2.0 Representative MassDOT UAS Mission and 
Cybersecurity Risk 


This section describes aerial mapping of approximately 10 acres that captured progress at the 
New MassDOT District 3 Administration Building construction site in Worcester, 
Massachusetts. Observing the stages of a MassDOT mission provided insight into the 
interactions between the UAS and its environment in order to enumerate sources of potential 
cybersecurity risk for this study. 


Figure 2.1 llustrates many of the factors associated with UAS mission. 


Energency checking Construction Monitoring 


| 


Mission Data 
Processing(Post-flight: 
checklists) 


Ongoing Mission 


Mission 
Planning(Pre- 
flight 
checklists) 


Real time checklists. 


] 
Cyber Physical security checking 


Risk Assessment 


‘ Resources PEN | Still images 
Security & Metadata 
° Check 


Site Description 


Zenmuse X4S . > 
Electo-Optical Resources 
Camera 
2 Qo 


Active 


: Work crew and 
DIIGO4 construction zone demonstration participants. Data Post-Processing 
] Crew members nas 4 
Drone Deploy (Flight | jon gaan moving SD cards containing flight 
Mission) | Multiple vehicies data 
construction vehicles 
Pix4D (Post-Processing Te buildi piacere ale Google Drive DPP server 
and Georectifying map) emporary Huang a 9 
Residential/Commercial Data retrieved, stored, 
Traffic Properties close to flight processed, and 
“ee areas disseminated 
= EMI from electric distribution Data uploaded to Pix4D & 
vo lines and construction Drone Deploy 
equipment 
Geo-rectified orthomosiac 
Pre-programmed Tailwind & rate of 15 minute and 14 Override the automated image 
ascentidescent seconds flight flight mission 
Automated flight mission VTON. location e-ight emergency Customer 


route Requirements Linear and volumetric 


calculations 


Figure 2.1; An example of MassDOT mission 


The thick black vertical bars denote the boundaries between the stages of the mission, including 
pre-mission (left), mission (center), and post-mission (right). Prior to the mission, hardware 
and software checklists are followed. The HeliPad at UMass Memorial Worcester was notified. 
An Inspire 2 UAS was equipped with a Zenmuse X4S Electro-Optical Camera, which can 
capture images with a ground sampling interval of 0.71 inches per pixel at an elevation of 200 
feet above ground level. 


Software applications to control the UAS: 
e Define a flight path, including altitude 
e Image post-processing 
e Map georectification included DJI GO 4 Drone Deploy, and Pix4D 


A default center defined where the aircraft will return to in case of failure or if the signal 
between the drone and pilot was lost. 


An aircraft can also perform obstacle detection if equipped with collision avoidance 
technology, such as vision, ultrasonic, infrared, and LIDAR sensors. The pilot can override the 
automated mission at any time. The inflight portion of the mission lasted approximately 15 
minutes. After ascent the UAS proceeded to follow the pre-programmed route to collect over 
300 NADIR pictures. The flight team included a remote pilot in command and a visual, 
observer monitored the mission site for dynamic hazards that could have been created by the 
motion of the UAS and obstacles such as an active construction zone, crew members, 
construction vehicles, temporary buildings, and traffic. During flight, integrated software 
algorithms and simultaneous localization and mapping (SLAM) technology constructed 3D 
maps on the pilot’s device, enabling the flight controller or pilot to sense and avoid objects. 
The ‘Internal Compass and Failsafe Function’ enables the UAS and pilot's remote-control 
system to precisely track its location. 'No Fly Zone Drone Technology’ can prevent unexpected 
flight patterns in constrained areas. 


Post-flight, images captured and their metadata were processed using high-performance 
computing and communications facilities to produce a geo-rectified orthomosaic image. 
Activities relevant to data security and privacy included handling of SD cards containing flight 
data and a Google Drive data processing program server for upload to Pix4D and Drone 
Deploy. 


2.1 Technical Decomposition of UAS Mission 
for Risks Identification 


Figure 2.2 shows the decomposition of a UAS mission, which is composed of one or more 
tasks that rely on functions. Each function is enabled by a combination of hardware, software, 
and communication and therefore a potential subject to attacks, posing corresponding risks. 
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3.0 Risk Assessment and Mitigation 


This section describes a UAS risk quantification approach and an objective strategy to mitigate 
risk through technology enhancement or countermeasures. 


Figure 3.1 illustrates the conceptual structure of the proposed UAS risk model. 
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Figure 3.1: Attack risk model of UASs 


Figure 3.1 contains eight nodes as well as the corresponding relations among them. It indicates 
that assets provide capabilities. However, they also possess vulnerabilities. Attacks target an 
asset through its vulnerabilities. Attacks transpire in the operational environment and are 
successful with a specified likelihood, producing consequences of a specified severity. The 
severity, likelihood, and operational environment contribute to risk. 


Table 3.1 lists a set of risk evaluation metrics for UAS. 


Table 3.1: Risk evaluation metrics for UASs 


Metric Description Range 

Impact How much damage can be caused | (0,1): where 0 means no impact and 1 asset 
by an attack. is completely compromised 

Likelihood The probability of successfully | (0,1): where 0 means impossible and 1 easy 
exploiting a vulnerability to exploit a vulnerability 


Impact and likelihood provide the basis for preliminary formulations of risk quantification and 
risk quantification. Table A.1 in the Appendices enumerates additional metrics that could 
further enrich the risk assessment and mitigation modeling presented here. 


3.1 Cyber Risk Enumeration Example 


Cyber risk management requires that risks and their potential consequences be identified. Only 
then is it possible to determine a strategy to mitigate these risks. 


Figure 3.2 illustrates the “Fly away” risk, where one of several attacks leads to the UAS flying 
away. 


Environment Goal Fly away 


Assets Drone Flight 


Threats 
Traffic injection, data 


manipulation, man in 
middle, fuzzing attack 


Sub-assets 1: : 
Mission data Tx (* 1c ountermeasures Sub-assets n: 
issranialata Ts Operational 


command Tx 


Vulnerabilities “148 
Vulnerabilities 

estimation estimation 
Likelihood Likelihood 


2S 


Risk = Likelihood x Impact 


é) 


2S 


Risk = Likelihood x Impact 


O 


Risk Evaluation = }\Likelihood x Impact 


Figure 3.2: Graphical cyber risk evaluation of “Fly away” 


The top center of Figure 3.2 indicates that the asset of the drone impacted is drone flight. Sub- 
assets required for this asset include transmission of mission data and command operations and 
that countermeasures that can reduce or potentially eliminate the impact of attacks on these 
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sub-assets. The top left of Figure 3.2 indicates the drone operating in an environment, which 
experiences a unique threat level and threats that could result in the drone flying away, such 
as: traffic injection, input data manipulation, and fuzzing as well as man-in-the-middle and 
potentially other attacks. Countermeasures of differing sophistication and cost can lower 
vulnerability and consequences of an attack, which can reduce the likelihood of an attack 
succeeding as well as its impact and corresponding severity. Traditional risk models quantify 
risk as the product of likelihood times impact. Assuming risks to sub-assets are mutually 
exclusive, allowing to sum over risk estimates for each sub-asset to obtain Risk(Fly away) = 
di Likelihood, x impact,, incorporating each threat specific to this risk. 


3.1.1 Risk Categories 


Table 3.2 lists 14 risks identified as part of this study as well as representative references. 


Table 3.2: Risk ID and name 


Risk Risk Name Ref. | Risk ID | Risk Name Ref. 
ID 

RO1 Fly Away (1) | RO8 Resource Leak (2) 
RO2 Loss of GPS (1) | ROS Battery Depletes (3) 
RO3 Loss of Data Link (1) | R10 Fuel Depletes (3) 
R04 Crash (1) | R11 Loss of Situational | (3) 

Awareness 

RO5 Autopilot Software Error/Fail (1) | R12 Loss of Direct Visual (3) 
RO6 GCS Failure (1) | R13 Hazard Weather (3) 
RO7 Automatic Transmission Locked | (4) | R14 Hostile Environment (3) 


Figure A.1 provides a graphical cyber risk assessment template, which identify the attacks 
associated. Tabular summarizes of Risks 1-6 are provided in Appendix B. Appendices C and 
D discuss attacks and countermeasures respectively. 


3.1.2 Risk Assessment Example 


Table 3.3 provides an example of risk assessment with respect to Fly away, indicating the 
attack by name, its likelihood, impact, and resulting risk; as well as the acceptability of this 
risk and recommendation regarding the urgency of mitigation. Note: the likelihood and impact 
are mission specific and have been assigned values here for the sake of illustration. 
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Table 3.3: “Fly away” cyber risk evaluation 


1. 


Attack | Attack Name_ | Likelihood | Impact | Risk | Acceptability | Recommendation 
ID 
A18 Man in the 3 2 6 | Tolerable Mitigate according to 
middle attack best practices 
All Communication 3 3 9 | Unacceptable | Immediate mitigation 
link jamming required 
A19 GPS jamming 3 2 6 | Tolerable Mitigate according to 
best practices 
A20 Replay attack 3 5 15 | Acceptable No action required 
A8 Sensor Spoofing 3 2 6 | Unacceptable | Immediate mitigation 
required 
AS Sensor Jamming 3 2 6 | Tolerable Mitigate according to 
best practices 
A brief description of the attacks underlying the fly away risk are as follows. 


Man in the middle attack (5) targets the communication between two components, 
typically client and server. Whenever one component attempts to communicate with 
the other to send data or authenticate, the attacker can observe and/or alter information 
before passing it to the other component. To overcome lack of trust in communication 
Common Attack Pattern Enumeration and Classification (CAPEC) recommended 
countermeasures include: 1) use of a Public Key signed by a Certificate Authority, 2) 
communication link encryption, 3) strong mutual authentication at both ends of any 
communications channel, and 4) exchange of public keys using a secure channel. 
Communication link jamming (6) prevents transmitting or receiving data from the 
targeted Wi-Fi network. Examples include: 1) flooding the Wi-Fi access point such as 
the retransmission device with de-authentication frames and 2) transmitting high levels 
of noise on the radio frequency band used by the Wi-Fi network. Countermeasures 
disassociate from flooding and radio frequency jamming, but are not standardized and 
must be supported on both the retransmission device and handset in order to be 
effective. 

GPS jamming (5) blocks all GPS communications, preventing the UAS from 
navigating. A simple type of attack is known as blanket jamming, which outputs noise 
or false information to saturate the GPS receiver. Countermeasures include 
retransmission and use of back up channels. 

Replay attack bypasses security by replaying a requests and can be performed in 
various ways. Countermeasures include an authentication mechanism that uses fresh 
message requests in a secure manner prior to data exchange or communication. 
Sensor spoofing (5) modifies original content, while keeping the source of the content 
unchanged. A sensor spoofing attack deceives the onboard UAS sensor regarding the 
environment or situation with the intention of misleading the UAS into taking an 
undesirable action. Countermeasures include verifying metadata along with the actual 
data and the use of redundant sensors (7). 

Sensor jamming (8) can deprive the UAS from information required to operate and act 
appropriately. Sensors may GPS-based navigation, a camera, IR sensor, barometer, 
which are also susceptible to jamming. The recommended countermeasure is sensor 
redundancy. 
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3.1.3 Cyber Risk Stoplight Charts 


A five-level cyber risk specification matrix of likelihoods and their coding (frequently (E), 
occasional (D), remote (C), improbable (B), and extremely improbable (A)) is provided 
because precise probabilities will be difficult to calculate. Thus, the proposed approach 
simplifies to categories in order to encourage adoption and elaboration. Similarly, impacts 
follow a five-level classification system and coding (extremely high (5), high (4), medium (3), 
low (2), and extremely low (1)), but this can also be adjusted according to the needs and 
practices of an organization. 


Table 3.4 color codes the combination of likelihood and impact indicates whether action is 
required (unacceptable (dark gray) tolerable (medium gray), and acceptable (light gray). 


Table 3.4: Cyber risk assessment matrix 


5 5A 5B 5C 5D 5E 
4 4A 4B AC 4D 4E 
= Ie 3A 3B 3C 3D 3E 
s | 2 2A 2B 2C 2D 2E 
—E f4 1A 1B 1C 1D 1E 
A B C D E 

Likelihood 


Table 3.5 provides recommendations according to the acceptability of the likelihood and 
impact. 


Table 3.5: Interpretation of cyber risk assessment matrix 


Acceptability | Likelihood/impact Recommendation 
Unacceptable | 3-5D and 1E-5E Immediate mitigation action and escalation is 
required. An operational stop should be considered 
Tolerable 4-5A, 3-5B, 1-5C, and | The cyber risk shall be mitigated as low as reasonable 
1-2D practicable and should a formal approval process 
followed. 
Acceptable 1-3A and 1-2B No action required. 


3.2. Countermeasure Portfolio Selection 
Problem 


This section develops a risk mitigation framework to allocate limited resources in a manner 
that reduces risk effectively. The proposed approach is a quantitative elaboration of the ‘Select’ 
step of the National Institute of Standards and Technology (NIST) Risk Management 
Framework (9) described in Section 0.2. 
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Consider a drone designed to perform M = M,,..., Mj, missions. Each mission is defined by 
a sequence of tasks that determine the hardware, software, and communication functionality 
the drone must possess to successfully execute that mission. Each function is vulnerable to one 
or more attacks, which pose corresponding risks. Without loss of generality, let A = Aj, ..., Ajq) 
be the set of all possible attacks that can be carried out against a drone, R = Ry,..., Rip; the 
risks posed by these attacks, and C = C;, ..., Cj¢, the countermeasures capable of mitigating or 
eliminating the impacts of the risks incurred by an attack. 


Not all attacks contribute to each risk. For example, risk RO1 (Fly Away) is susceptible to six 
attacks, namely: man in the middle (A18), communication link jamming (A11), GPS jamming 
(A19), replay attack (A20), sensor spoofing (A08), and sensor jamming (A09), which we 
denote Attacks(RO2)={A08, AO9, A11, A18, A19, A20} or Attacks(R;) more generally. 
Similarly, we denote the countermeasures capable of mitigating the ith attack as 
Countermeasures (A;). 


Each attack has a corresponding probability (Likelihood) of occurrence (Pr{A;}) as well as a 
corresponding impact (Impact(A;)), which is conditional upon the subset of countermeasures 
C’ € C, such that the impact of a risk with respect to its corresponding attacks and their impact 
conditioned on the countermeasures is Impact(R;,) = XL, Pr{A;}Impact(A;|C’), where 
Impact(A;|C,) < Impact(A;|C,;) when C,€C;,, meaning that adding additional 
countermeasures decreases the impact. The overall impact of a set of risk R’ € R is therefore 
the sum of the risks to which the drone is susceptible Impact(R’) = Yip'er Impact (Rj) 


This specification enables the definition of the countermeasure selection problem as the 
following budget constrained optimization problem 


Minimize Impact(R’) (1) 
Subject to 


I(C; € C’) x Cost(€;) < B (2) 
C'EC 


where the indicator function I[(C; € C’) = 1 if countermeasure C; is in the set of selected 
countermeasures and Cost(C;) is the cost of the ith countermeasure. 


Inclusion of a countermeasure in a portfolio is a binary decision, and can, therefore, be 
represented as a binary string of length |C|, where a p; = 1 if the counter measure is in the 
portfolio and 0 otherwise. Problems such as these can be solved effectively with methods such 
as the genetic algorithm (GA) and the solution. In the case where the drone performs more than 
one mission and some risks are unique to M, or Mg, there is no single optimum countermeasure 
portfolio to reduce impacts to both missions, requiring a multi-objective solution that gives 
rise to a Pareto optimal front of solutions, where reducing the impact of M, may adversely 
affect the impact with respect to M, and vice versa. 


Figure 3.3: provides a graphical representation of the countermeasure portfolio problem with 
respect to a single mission. 
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UAV 


J 


Mission (M1) 


Figure 3.3: Graphical representation of countermeasure portfolio selection 


A mission is subject to risks according to the capabilities required for that mission (dot-dot- 
dash arrows). Risk are posed by one or more underlying attacks (dashed arrows), as discussed 
in the previous section. Countermeasures (solid arrows) can mitigate one or more distinct 
attack and some attacks may contribute to more than one risk. Primary challenges are to 
elaborate mission specific risk and the effectiveness of countermeasures, after which it is 
possible to make an informed judgment regarding the countermeasures that should be taken 
proportional to risk appetite and budget constraints. 


3.3 Countermeasure selection Illustration 


We illustrate countermeasure selection in the context of flyaway risk (Risks RO1), which may 
be caused by Attacks(R01)={A08, AO09, A11, A18, A19, A20} given in Table 3.3 and counter- 
measure Countermeasures(Attacks(RO1))= C2,C8,C9,C10,C13,C14,C19} given in 
Appendix F. Table D.2 provides a graphical representation of the the fly away risk as well as 
the underlying attacks and potential countermeasures. 


UAV «— Risk 
+-- Attack 
+— Countermeasure 


(0,-2,$5) (0,-1,$10) (0,-3,$5) (0,-2,$1) (0,-1,$1) (0,-1,$15) (0,-2,$1) 


Figure 3.4: Fly away risk attack countermeasure dependencies 
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Tuples (L;,1;) attacks (Table 3.6) indicate the likelihood and impact of that attack, while the 
tuples (CJ;,C,) indicate the reduction in likelihood of impact and corresponding cost of a 
countermeasure. A negative value indicate a reduction in likelihood (deterrent) and/or impact 
(mitigation). 


Table 3.6 summarizes the attack, risk, and countermeasure information for clarity. 


Table 3.6: Various measures for attack and countermeasure 


Attack (A;) L;| 1,| Risk | Countermeasures (C;) CI; C, 
gl fell) 4g Verify metadata along with actual 
A8 Sensor Spoofing C8 data. 0,-1 $10.00 
313 9 Cross verify data from redundant 
A9 Sensor Jamming C9 sensors 0,-1 $5.00 
C10 (0,- 
Communication 3] 2 6 C 13 Measure signal power level to detect | 1), $1, 
All| link jamming jamming, Channel switching (0,-2) | $1 
Man in the middle 3/5]| 415 
A18 | attack C2 Utilize strong federated identity 0,-2 $5.00 
A19_ | GPS jamming 3 | 2 6 C14 __| Use backup channels; 0,-1 $15.00 
314 3 Use secure and robust protocols with 
A20_| Replay attack C19 strong authentication 0,-2 $1.00 
Total Risk ol 


3.3.1 Cost of Implementing Countermeasures 


The goal is to identify countermeasures that reduce risk to an acceptable level as indicated in 
Table 3.3 in order to achieve a level of cyber risk acceptance as defined in Table 3.4 and Table 
3.5. 


With no countermeasures allocated, the baseline fly away risk is 
Ro = (4X 3)+(8X3)4+(8X2)+(3X5)+(3 X2)4+ (3X1) =51, 
whereas implementing a counter measure such as C2 reduces risk to 
R(C2) = (4% 3)+ (3X3) + (3X 2) + (8-2) x5)+(8x2)+ 8X1) =45 
and the effective risk to cost ratio is 
Ro —R(C2) 51-45 
eae = 12 


Similarly the risk reduction for the other countermeasures are: R(C8) = 48, R(C9) = 48, 
R(C10) =48, R(C13) = 45, R(C14) = 48, and R(C19) = 45. 


RC, = 


Table 3.7 reports the risk to cost ratio of all countermeasures. 


Table 3.7: Effective risk to cost ratio for countermeasure impact 


Countermeasure Ratio 
C2 1.200 
C8 0.300 
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Countermeasure Ratio 
C9 0.600 
C10 3.000 
C13 6.000 
C14 0.200 
C19 6.000 


Both C13 and C19 possess the same cost ratio. However, A11 has a higher impact than A20, 
so C13 for A11 is selected. The ratios are then recomputing with the new total risk baseline of 
45. Figure 3.5 shows the total risk as a function of the cumulative cost of countermeasures. 


60 


Total risk 


10 15 20 
Cost 


Figure 3.5: Risk reduction Pareto front 


This approach enables a decision-maker to identify the risk attainable within a specified budget 
or the cost required to achieve a desired risk level. This approach can guide countermeasure 
selection as well as support budget justifications for such countermeasures. 


Table 3.8 provides the details of the iterations of a greedy algorithm to allocate 
countermeasures, which produced Figure 3.5 above. 


Table 3.8: Iterations of greedy algorithm for countermeasure selection 


Cost | Risk reduction | Countermeasure subset Selected Countermeasure 
0 51 {C8,C9,C10,C13,C2,C14,C19} 0 

1 45 {C8,C9,C10,C2,C14,C19} C13 

2 42 {C8,C9,C10,C2,C14} C19 

3 42 {C8,C9,C2,C14} C10 

8 27 {C8,C9 ,C14} C2 

13 18 {C8,C14} C9 
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Cost | Risk reduction Countermeasure subset Selected Countermeasure 
23 6 {C14} C8 
38 0 {0} C14 


A more fine-grained approach can consider multiple dimensions by quantifying risk with 
respect to the failure modes and effects of the various attacks, which would enable 
countermeasure selection to reduce risk with respect to multiple categories of consequences. 
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4.0 Standards 


The ANSI (American National Standards Institute) Unmanned Aerial Systems Standardization 
Collaborative (UASSC) has developed standardization roadmap (10) and maintains links to 
UAS Standards (11). Of these links, the American Society for Testing and Materials, ASTM 
F3201-16 Standard Practice for Ensuring Dependability of Software Used in Unmanned Aerial 
Systems (UAS) is most relevant to this study, especially security as an enabler of safety. The 
remainder of this section summarizes prominent standards in the areas of aerial systems safety, 
UAS navigation and communication, and cyber test and evaluation, and cyber risk 
management. It is suggested that, before endorsing standards for use within the 
Commonwealth of Massachusetts, MassDOT should ensure that software and cybersecurity 
experts provide input on standards to ensure that these standards reference relevant 
cybersecurity standards and best practices, and that they are kept up to date on a regular basis. 


4.1 Aerial Systems Safety 


4.1.1 Safety: Department of Defense Standard Practice System Safety 882E 


MIL-STD-882E is relevant because safety is a concern of MassDOT and cybersecurity 
vulnerabilities pose threats to system safety. MIL-STD-882E (12) identifies the Department of 
Defense (DoD) approach for identifying hazards, assessing and mitigating associated risks 
encountered in the development, test, production, use, and disposal of defense systems MIL- 
STD-882E defines the risk acceptance authorities. It also defines the system safety 
requirements throughout the life-cycle for any system and when properly applied, these 
requirements should enable the identification and management of hazards and their associated 
risks during system development and engineering sustainment activities. MIL-STD-882E 
provides four different severity categories starting from a loss of a work day to severe 
environmental impact, potential death or permanent disability. This standard also categorizes 
the hazard at a given point of time such as the probability of the hazards. A unified risk 
assessment matrix is provided. 


The system safety process consists of managing life-cycle risk, software contribution to system 
risk, and software assessment. Software safety criticality matrix maps the software controls to 
severity categories using software criticality indices (SwC]). Task 102 system safety program 
develops a plan to document the system safety methodology for the identification, 
classification, and mitigation of safety hazards as part of the overall systems engineering 
process. 


MIL-STD-882E also provides software system safety engineering and analysis requirements. 
This standard mentions that (12) “from the perspective of the system safety engineer and the 
hazard analysis process, software is considered as a subsystem.” System safety engineers 
should ensure that software is considered in its contribution to mishap occurrences for the 
system under analysis, as well as interfacing systems within a systems of systems architecture. 
The software system safety processes and requirements are based on the identification and 
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establishment of specific and test tasks for each acquisition phase of the software development 
life-cycle. The software risk assessment should follow the same risk criteria or risk matrix as 
hardware system. 


4.1.2 Airborne Systems: DO-178C Software Considerations in Airborne Systems and 
Equipment Certification 


UAS are aerial systems and therefore are covered by DO-178C (13), which is an RCTA (Radio 
Technical Commission for Aeronautics) standard for demonstrating compliance with 
applicable airworthiness regulations for software aspects of aerial systems and equipment 
certification. 


DO-178C consists of software considerations in Airborne Systems and Equipment 
Certification, published by RTCA. This standard categorizes the software into five hazard 
levels based on System Safety Assessment: 

e Level A hazards consists of anomalous behavior of the aerial system resulting in 
catastrophic failure condition. These types of behavior prevent continued safe flight 
and landing. 

e Level B hazards affects safety-critical capabilities and can result in serious or 
potentially fatal injuries. 

e Level C hazards produce a major failure condition, where the hazard results in 
discomfort to occupants, possibly including injuries. 

e Level D hazards result in a minor failure condition and some inconvenience to 
occupants. 

e Level E hazards correspond to safe operational conditions and result in no effect on 
aircraft operational capability or the pilot. 


DO-178C supports the objective verification of output of the software coding and integration 
process. The recent version of the standard also considers economic impact relative to system 
certification without compromising system safety. The primary steps for the software safety 
certification consists of formal methods for verification, object oriented technology, model 
based development and verification, and tool qualification. 


4.2 UAS Navigation and Communication 


4.2.1 Navigation: RTCA/DO236B Minimum Aviation System Performance Standards 


RTCA/DO-236B (14) defines the path the aircraft must use to evaluate performance. The 
aircraft’s navigation system will also define all vertical paths in the Final Approach Segment 
(FAS) by a Flight Path Angle (FPA) as a trajectory to a fix and altitude. However, RTCA/DO- 
236B facilitates airspace design and does not directly equate to obstacle clearance. 
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4.2.2 Communication: [EEE 1609 - Family of Standards for Wireless Access in Vehicular 
Environments (WAVE) 


The IEEE 1609 Family of Standards (15) includes several active sub-standards related to cyber 
security of UAS communication: 
1. P1609.0 - IEEE Draft Guide for Wireless Access in Vehicular Environments (WAVE) 
- Architecture 
2. P1609.2b - Standard for Wireless Access in Vehicular Environments--Security 
Services for Applications and Management Messages Amendment 2: Protocol Data 
Unit (PDU) Functional Types and Encryption Key Management 
3. 1609.0-2013 - IEEE Guide for Wireless Access in Vehicular Environments (WAVE) - 
Architecture 
4. 1609.2-2016 - IEEE Standard for Wireless Access in Vehicular Environments-- 
Security Services for Applications and Management Messages 
5. 1609.4-2016 - IEEE Standard for Wireless Access in Vehicular Environments 
(WAVE) -- Multi-Channel Operation 
6. 1609.2a-2017 - IEEE Standard for Wireless Access in Vehicular Environments-- 
Security Services for Applications and Management Messages - Amendment 1 


4.3 Cyber Test and Evaluation and Risk 
Management 


4.3.1 Cyber Test & Evaluation: Department of Defense Cyber Test and Evaluation 
Guidebook Version 2.0 


The Cyber Test and Evaluation Guidebook (16) develops data-driven mission-impact-based 
analysis and assessment methods for cybersecurity test and evaluation (T&E) and supports 
assessment of cybersecurity, survivability, and resilience within a mission context by 
encouraging planning for tighter integration with traditional system T&E. Cyber-security T&E 
starts at acquisition initiation and continues throughout the entire life cycle. A primary 
objective for test and evaluation is to understand how adversarial attacks affect a cyber physical 
system and the missions it is designed to perform. 


Cybersecurity T&E consists of six phases aligned with DOD 15000.02 Operation of the 
Defense Acquisition System: 

1. Phase 1 examines a system’s cybersecurity and resilience requirements in order to 
develop an initial approach and plan for conducting cybersecurity T&E. This phase is 
performed during the early design and planning lifecycle. 

2. Phase 2 characterizes the attack surface, identifies the vulnerabilities, and avenues of 
attack an adversary may use to exploit the system. This phase develops the plans to 
evaluate the impact of attacks on the mission. 

3. Phase 3 verifies the cybersecurity and needed counter-measures, which helps 
stakeholders and designers reduce risk. This phase is conducted during developmental 
test and evaluation. 

4. Phase 4 performs adversarial tests in the context of mission operations to identify 
residual risks. 
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3. 


Phase 5 characterizes the cybersecurity and resilience status of a system in a fully 
operational context and provides reconnaissance on the system. This phase is 
conducted during operational test and evaluation. 

Phase 6 characterizes the operational mission effects to critical missions caused by 
threat-representative cyber activity against a unit trained and equipped with a system 
as well as the effectiveness of defensive capabilities. This phase is also performed 
during operational test and evaluation. 


4.3.2 Cyber Risk Management: National Institute of Standards and Technology Risk 
Management Framework (RMF) 


The NIST Risk Management Framework (9) integrates risk management into the system 
development lifecycle. It offers a holistic framework and process for determining 
organizational, mission, and system risk. The framework consists of six steps and three level 
of organization wide risk management. The six steps are: 


Ls 


Categorize: The purpose of this stage is to determine the order of risk criticality and its 
impact on the organization, mission, or system. 

Select: This step selects various security controls or countermeasures based on the 
outputs from Step 1. A risk assessment is performed in this stage. A baseline risk or 
threat level is also specified. 

Implement: In this step, the security control or the various countermeasures are 
implemented within the system or enterprise architecture. 

Assess: This step determines the effectiveness of the security measures implemented, 
operational effectiveness, and requirements. This step determines the depth and 
coverage needed for system assurance. Both hardware and software risk assessment 
should be conducted. 

Authorize: In this step, an expert examines the output of step four to determine the 
effectiveness of the risk management framework implementation. 

Monitor: The final step involves the continuous monitoring of the system and its 
operational environment for changes or sign of attack. Monitoring activities should be 
integrated into the organization network wide. 
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10. 


11. 


12. 


13. 


14. 
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6.0 Appendices 


Appendix A: Additional Risk Evaluation 


SS 
Table A.1: Graphical cyber risk assessment template 
Metric Description Value 
Asset Capacity To what level the asset was (0,1): 0 means the asset is 
compromised under an attack totally compromised, 1 means 
fully operational 
Number of Attack Paths The number of potential attack n: the number of potential 


paths in a network. 


attack paths 


Operational Capacity 


The remained operation capacity 
of a system after being attacked 


(0,1): 0 means not operational, 
1 means fully operational 


Service Availability 


The availability of a required 
service to support a particular 
mission 


{0, 1}: 0 means service is not 
available, 1 means service is 
available 


Severity Score 


The severity of a vulnerability if it 
was successfully exploited, could 
be measure based on CVSS 
(Common Vulnerability Scoring 
System) score (outcome). 


(0,1): O means no risk, 1 means 
high risk. 


Graphical Cyber Risk Assessment Template 


Figure A.1 is an abstraction of the “Fly away” risk. It enables the graphical summarization of 
the threats specific to a cyber-risk in the form of a template. 
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Goal 
Security objective to 


prevent risk 
Environment | 
Assets a he = 
Threat Level/Cost (systems/functions/ ae 
protocols/components) 


Threats ———oe Sub-assets i¢<————_—— Countermeasures 


attack successful 


Vulnerabilities | Consequences 
Estimation 
Likelihood per Impact Level 
successful attack Estimation 


Risk Estimation 


= py Likelihood xX Impact 
i 


Figure A.1: Graphical cyber risk assessment template 


More generally, assets operate in environments, which experience threat levels and therefore 
pose threats to the sub-assets of an asset. Each sub-asset can be safeguarded with 
countermeasures to lower the likelihood of a successful attack and potentially the impact of a 
successful attack. Thus, asset risk is expressed as Risk = )); Likelihood; : Impact;, where i 
represents i‘ sub-asset. 
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Appendix B: Risks 


Risk 1: Fly Away (Operation Phase: Flight & Pre-Flight Operation) 
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Table B.1 identifies the attacks associated with the risk. It also summarizes risk assessment 
and countermeasures. Attack types are categorized according to the CAPEC and attack 
mechanisms and the components target specified. Risk assessment specifies the likelihood and 
impact of attacks. Current and recommended actions to mitigate risk are also identified. 


29 


Table B.1: Fly away 


Attack Identification 


Attack Types (CAPEC): 


Attack Mechanisms 
(Attack ID #, Mechanisms, 


Types): 


Communication Channel Manipulation; Obstruction; Protocol 
Manipulation; 


1.1 Man in the middle attack (Communication Channel 
Manipulation) 

1.2 Communication link jamming (Obstruction) 

1.3 GPS jamming (Obstruction) 

1.4 Replay attack (Protocol Manipulation) 

1.5 Sensor Spoofing (Protocol Manipulation) 

1.6 Sensor Jamming (Obstruction) 


Components Targeted 
(Components & Attack ID #): 


Hardware Components: 
GPS sensor (ID: 1.5, 1.6); Camera sensor (ID: 1.5, 1.6); 
Obstacle avoidance sensors (ID: 1.5, 1.6) 


Functional Components (software/algorithm + protocol): 
Mission data transmission link (ID: 1.1, 1.2, 1.4); GPS 
transmission link (ID: 1.3) 


Lik 


ie 


elihood, Impact, and Risk Assessment 


Likelihood of attack 
(Likelihood & Attack ID #) 


Frequently 
Occasional (ID: 1.2, 1.3, 1.6) 


Remote (ID: 1.1, 1.4, 1.5) 


Improbable 
Extremely Improbable 


Impact of successful attack 
(Impact level & Attack ID #): 


Risk Level 
(Risk level & Attack ID #): 


High (ID: 1.1-1.4) 


Medium (ID: 1.5, 1.6) 


Low 


Unacceptable 
Tolerable 
Acceptable 


Coun 


termeasure-oriented Recommendations 


Recommendations/actions 
(in literatures and COTs) for 
given drone 
configuration/operation 
practice: 


(Recommendations & Attack 
ID # & Targeted components) 


Current actions: 

Checklist activities for handling emergency conditions 
(ID: 1.1-1.6) 

Manual monitoring of the flight (ID: 1.1-1.6) 


Recommended techniques: 

Authentication and Encryption mechanisms (ID: 1.1) on 
mission data link; 

Anti-jamming techniques (ID: 1.2, 1.3) on mission data 
link, GPS transmission link; 

Verify metadata along with actual data (ID: 1.4, 1.5, 1.6) 
on mission data link, GPS sensor, Camera sensor and 
Obstacle avoidance sensors 
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Risk 2: Loss of GPS 


Table B.2: Loss of GPS 


Metric Category 


Category Option Chosen for Current Practice 


ID #: 


2 


Operation Phase 


Flight Operation 


Attack Types (CAPEC) 
Attack Mechanisms 
(Attack ID # & Attack 
Mechanisms & Attack 


Types) 


Attack Identification 


Obstruction; Protocol Manipulation; Interception 


2.1 GPS Spoofing (Protocol Manipulation) 
2.2 GPS jamming (Obstruction) 
2.3 Sensor Sniffing (Interception) 


Components Targeted 
(Components & Attack ID 
#) 


Hardware Components: 

GPS sensor (ID: 2.1-2.3); Camera sensor (ID: 2.1-2.3); 
Obstacle avoidance sensors (ID: 2.1-2.3); Magnetometer 
(ID: 2.1-2.3) 


Functional Components (software/algorithm + protocol): 
GPS signal transmission (ID: 2.1-2.3) 


Safety Risk Assessment 

Likelihood of attack e Frequently 
(Likelihood & Attack ID #) e Occasional (ID: 2.1, 2.2, 2.3) 

e Remote 

e Improbable 

e Extremely Improbable 
Impact level of successful e High 
attack (Impact level & e Medium (ID: 2.1, 2.2, 2.3) 
Attack ID #) e Low 
Risk Level e Unacceptable 
(Risk level & Attack ID #) e = Tolerable 

e Acceptable 


Countermeasure-oriented Recommendations 


Recommendations/actions 
(in literatures and COTs) for 
given drone 
configuration/operation 
practice 


(Recommendations & 
Attack ID # & Targeted 
components) 


Current actions: 

Checklist activities for handling emergency conditions (ID: 
2.1-2.3) 

Manual monitoring of the flight (ID: 2.1-2.3) 


Recommended techniques: 

Verify metadata along with actual data (ID: 2.1) on GPS 
sensor, Camera sensor and Obstacle avoidance sensors and 
Magnetometer 

Measure the signal power level to detect jamming (ID: 2.2) 
on GPS sensor, Camera sensor and Obstacle avoidance 
sensors and Magnetometer 

Use efficient cryptographic techniques, lie keys stream, 
one-time key (ID: 2.3) on GPS sensor, Camera sensor and 
Obstacle avoidance sensors and Magnetometer 
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Risk 3: Loss of Data Link 


Table B.2: Loss of Data Link 


Metric Category 


Category Option Chosen for Current 


Practice 
ID #: 3 
Operation Phase Flight Operation 
Attack Identification 


Attack Types (CAPEC) 


Obstruction; Communication Channel 
Manipulation; Obstruction 


Attack Mechanisms 
(Attack ID # & Attack Mechanisms & Attack 
Types) 


3.1 Communication link jamming 
(Obstruction) 

3.2 Man in the middle attack 
(Communication Channel 
Manipulation) 

3.3 GPS jamming (Obstruction) 


Components Targeted (Components & Attack 
ID #) 


Functional Components 
(software/algorithm + protocol): 
Mission data link (ID: 3.1, 3.2); GPS 
transmission link (ID: 3.3) 


Safety Risk Assessment 


Likelihood of attack (Likelihood & Attack ID #) 


Frequently 


Occasional 
Remote (ID: 3.1, 3.3) 


Improbable (ID: 3.2) 


Extremely Improbable 


Impact level of successful attack e High 
(Impact level & Attack ID #) e Medium (ID: 3.1-3.3) 
e Low 
e Catastrophic 
Severity of attack e Hazardous 
(Severity & Attack ID #) e Major (ID: 3.1-3.3) 
e Minor 
e Negligible 
Risk Level e Unacceptable 
(Risk level & Attack ID #) e Tolerable 
e Acceptable 


Countermeasure-oriented Recommendations 


Recommendations/actions (in literatures and 
COTs) for given drone configuration/operation 
practice 


(continued on next page) 


Current actions: 

Checklist activities for handling 
emergency conditions (ID: 3.1-3.3) 
Manual monitoring of the flight (ID: 
3.1-3.3) 
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Metric Category 


Category Option Chosen for Current 
Practice 


ID #: 
Operation Phase 


3 


Flight Operation 


(Recommendations & Attack ID # & Targeted 
components) 


e Recommended techniques: 

e = Anti-jamming techniques (ID: 3.1, 3.3) 
on mission data link, GPS transmission 
link; 

e Authentication and Encryption 
mechanisms (ID: 3.2) on mission data 
link; 
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Risk 4: Crash 


Table B.3: Crash 


Metric Category 


Category Option Chosen 


ID # 


4 


Operation Phase 


Flight Operation & Pre-Flight Operation 


Potential Attack Types 
(ID # & Attack Type) 


4.1 Hardware Integrity Attack 

4.2 Forced Deadlock 

4.3 Malicious Logic Insertion 

4.4 Fault Injection 

4.5 Exploiting Trust in Client 

4.6 Authentication Bypass 

4.7 Communication link jamming (17) 
4.8 GPS jamming (17) (Interception) 
4.9 Replay attack (17) 

4.10 Sensor Spoofing (17) 

4.11 Sensor Jamming (17) 


Countermeasures 


Component Affected 


Checklist activities for handling emergency conditions (ID: 4.1- 
4.11) 

Manual monitoring of the flight (ID: 4.1-4.11) 

Verify metadata along with actual data (ID: 4.9, 4.10) 
Anti-jamming techniques (ID: 4.7, 4.8, 4.11) 

Encryption and sensor firmware robustness (ID: 4.10) 
Hardware Components: 

Electronic Speed Control Circuits (ESCs) (ID: 4.1, 4.4); Motors 
(ID: 4.1) 


Functional Components: 
Central Flight Controller (ID: 4.1-4.5); Environmental 
Perception (ID: 4.2, 4.4, 4.6); Auto Pilot (ID: 4.2, 4.4, 4.6) 


Consequences 


Harm to people 
Damage of drone 
Damage of infrastructure 


Likelihood 


Frequently 


Occasional (ID: 4.7-4.11) 


Remote (ID: 4.1-4.6) 
Improbable 


Extremely Improbable 


Impact 


High 
Medium (ID: 4.1-4.11) 


Low 


Final Recommendation 
(Risk Level) 


Unacceptable 
Tolerable 
Acceptable 
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Risk 5: Autopilot Software Error/Fail 


Table B.4: Autopilot Software Error/Fail 


Metric Category 


Category Option Chosen 


ID # 


p) 


Operation Phase 


Flight Operation & Pre-Flight Operation 


Potential Attack Types 
(ID # & Attack Type) 


5.1 Signal Integrity (Command Injection) 
5.2 Hijacking 

5.3 Malwares (5) (Contaminate Resource) 
5.4 Code Injection (6; 5) 

5.5 Hardware Integrity Attack 

5.6 Fault Injection 

5.7 Command Injection (6; 5) 

5.8 Man in the middle attack (17; 18) 

5.9 GPS jamming (6; 5) (Interception) 
5.10 Replay attack (6; 5) 


Countermeasures 


Checklist activities for handling emergency conditions (ID: 5.1- 
5.10) 

Manual monitoring of the flight (ID: 5.1-5.10) 

Verify metadata along with actual data (ID: 4.9, 4.10) 
Anti-jamming techniques (ID: 4.7, 4.8, 4.11) 

Encryption and sensor firmware robustness (ID: 4.10) 


Component Affected 


Consequences 


Hardware Components: 
Autopilot (ID: 5.3); 


Software Components: 
Cyber (ID: 5.3) 


Functional Components: 

Mission data link (ID: 5.2, 5.8-5.10); Operation Command Link 
(ID: 5.2, 5.7, 5.8, 5.10) 

Harm to people 

Damage of drone 

Damage of infrastructure 


Likelihood 


Frequently 
Occasional 


Remote (ID: 5.1-4.6) 


Improbable (ID: 5.7-5.10) 


Extremely Improbable 


Impact 


High 


Medium (ID: 5.1-5.10) 


Low 


Final Recommendation 
(Risk Level) 


Unacceptable 
Tolerable 
Acceptable 
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Risk 6: GCS Failure 


Table B.5: GCS Failure 


Metric Category Category Option Chosen 
ID# 6 
Operation Phase Flight Operation 
Potential Attack Types e 6.1 Spyware (6; 5) 
(ID # & Attack Type) e 6.2 Malware (6; 5) 
e 6.3 Authentication Bypass 
e 6.4 Exploiting Trust in Client 
e 6.5 Interception 
e 6.6 Infrastructure Manipulation 
Countermeasures e Checklist activities for handling emergency conditions (ID: 6.1- 
6.6) 
e Manual monitoring of the flight (ID: 6.1-6.6) 
e Anti-spyware software, firewalls, packet filters [7] (ID: 6.1) 
e Anti-malware software, packet filters, firewalls [7] (ID: 6.2) 
Component Affected e Hardware Components: 
e GCS (ID: 6.1, 6.2) 
e Functional Components: 
e GPS transmission (ID: 6.3-6.6) 
Consequences e Harm to people 
e Damage of drone 
e Damage of infrastructure 
Likelihood e Frequently 
e Occasional 
e Remote (ID: 6.3-6.6) 
e Improbable (ID: 6.1-6.2) 
e Extremely Improbable 
e High (ID: 6.3-6.6) 
Impact e Medium (ID: 6.1-6.2) 
e Low 
Final Recommendation e Unacceptable 
(Risk Level) e Tolerable 
e Acceptable 
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Appendix C: Attacks 


Table C.1 provides attack ID, the attack name, links to the CAPEC (Common Attack Pattern 
Enumeration and Classification) ID wherever possible, and references. 


Table C.1: List of attack mechanisms 


Attack Attack Name Ref. Attack Attack Name Ref. 
ID (CAPEC ID) ID (CAPEC ID) 

A0l1 Code Injection (242) (18; 19) A23 Rogue Node (616, 524) (18) 

A02 Identity Spoofing (151) (7) A24 Theft and Vandalism (507) (7) 

A03 Sleep Deprivation (18; 2) A25 Rogue Drone Collision (20) 

Attack 

A04 Hardware Integrity Attack | (20) A26 Firmware Modification (638) | (18) 
(440) 

A05 Fault Injection Attack (21) A27 Supply Chain Attack (18) 
(624) (522,544) 

A06 Spyware (549) (19) A28 Corruption (22) 

AO07 Malwares (441) (23) A29 Video Replay Attack (19) 

A08 Sensor Spoofing (148) (2; 23) A30 Root Kits (552) (18) 

AO09 Sensor Jamming (601) (8) A31 Key Loggers (568) (18) 

A10 Sensor Sniffing (157) (19) A32 Password Cracking (55) (18) 

All Communication Link (18) A33 Eavesdropping (651) (7) 
Jamming (601) 

A12 Command Injection (248) | (18; 24) A34 Scrambling/Distortion 

A13 False Data Injection (240) | (18; 23) A35 Reference Station Attack (18) 

Al14 Fuzzing Attack (28) (18; 20) A36 Signal Delay (236) (18) 

A15 Network Isolation (18) A37 Address Resolution Protocol | (19) 

(S90) 

A16 Black Hole/Gray Hole (18) A38 Hijacking (501) (18) 

A17 Packet Sniffing (157) (18) A39 Cross Layer Attack (18) 

A18 Man in the Middle Attack | (18) A40 Multi-Protocol Attack (7) 
(94) 

A19 GPS Signals Jamming (18; 25) A41 Back Doors (18) 
(627) 

A20 Replay Attack (60) (18; 19) A42 Code Modification (242) (19) 

A21 Denial of Service (210) (7; 25) A43 Extemal Signal Spoofing (19) 

A22 De-authentication Attack (19) A44 In-Vehicle Spoofing (19) 


Selected literature on UAS testing, risk modeling, and architectural frameworks 


The cybersecurity literature is vast. Therefore, this selected literature review describes past 
studies performed in the context of UAS. Specifically, testing, risk modeling and mitigation, 
and architectural frameworks are considered. Papers focused on a single UAS attack are not 
discussed here, but can be identified from the references. Both attacks and countermeasures 


are covered because a comprehensive method to design and test require both perspectives. 


af 


Altway and Youssef (22) identified security, safety, and privacy aspects of civilian drone 
operation, including protocols for fail-safe procedures. Horowitz et al. (26) developed 
architectural decision support, mission-centric analysis and modeling methods to combining 
inputs from system experts at the design and user levels in support of cybersecurity aware 
systems engineering. The thesis of Leccadito (18) developed a hierarchical embedded cyber 
attack detection framework in the context of UAS to ensure security is a testable property that 
is built into a system. Hagerman et al. (27) described a UAS security testing approach which 
uses behavioral, attack, and mitigation models. The behavioral and attack models are used to 
identify attack points. The mitigation model then generates the security test suite. DroneJack 
(28) allows the user to shutdown a UAS, pilot the UAS, or direct it to GPS coordinates as well 
as exploit data, including recovery of photo, video, and flight logs. Custom attacks can also be 
configured and deployed. DroneJack can therefore be used as a testing tool. The Defense 
Advanced Research Projects Agency (DARPA) High Assurance Cyber Military Systems 
(HACMS) Program demonstrated that an open-source quadcopter was secured from hijacking 
despite being given six weeks and full access to the source code of the copter. 


Krishna and Murphy (19) reviewed UAS cybersecurity vulnerabilities and developed a UAS 
specific taxonomy of attacks according to attack vector and target. Solodov et al. (29) 
overviewed UAS technology and potential threats to nuclear facilities, evaluating measures to 
detect, delay, and neutralize. Mansfield et al. (30) analyzed security vulnerabilities within 
smart phones and tablets, and software applications to develop a risk model of the threat profile 
of the Department of Defense Ground Control Station communications. Similarly, Hartmann 
and Steup (8) developed a risk assessment methodology for UAS which considers physical and 
environmental factors, communication, storage media, sensors, and fault handling 
mechanisms. 


Garg et al. (31) anticipate the role of UAS will play in edge computing, which creates new 
quality of service requirements to ensure uninterrupted data sharing for what may come to be 
regarded as mission and life critical services. Toward this end, the authors proposed a data- 
driven transportation optimization model that also conducts cyber-threat detection. 
Vattapparamban et al. (32) review aspects of UAS related to cybersecurity, privacy, and public 
safety. They also provide examples of attacks on UAS as well as UAS as a platform from 
which to carry out attacks. Kim et al. (6) reviewed general and network systems specific 
cyberattacks to identify potential threats, vulnerabilities, post-attack behaviors in existing 
autopilot systems. Stracquodaine et al. (2) presented a comprehensive method to protect a UAS 
from hardware and software attacks by directly monitoring the autopilot as well as the onboard 
operating system, enabling dependable operation despite sensor or data spoofing attacks. Given 
the high potential for cyberattacks on UAS, Blazy et al. (21) proposed an efficient protocol to 
ensure the confidentiality of data collected, which is independent of the encryption scheme 
implemented. 
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Appendix D: Countermeasures 


Table D.1 lists classes of countermeasure identified according to their [ID and name. 


Table D.1: Countermeasures 


Countermeasure 
ID Name ID Name 
C01 | Update firmware C13 | Use channel switching 
C02 | Utilize strong federated identity | C14 | Use backup channels 
C03 | Check for power leakage C15 | Perform jamming detection techniques 
C04 | Use intrusion detection C16 | Validate user- controllable input 
technique 
C05 | Perform penetration testing C17 | Whitelisting/blacklisting the inputs. 
C06 | Use anti-spyware and packet C18 | Use Fault detection approach 
filters 
C07 | Use firewalls, anti-malware and | C19 | Use secure and robust protocols with 
packet filters strong authentication 
C08 | Verify metadata along with C20 | Utilize redundant communication links 
actual data. 
C09 | Cross verify data from C21 | Utilize strong passwords 
redundant sensors 
C10 | Measure the signal power level | C22 | Fail safe/fail loud protocol 
to detect jamming 
C11 | Use encryption technique C23 | Use physical security techniques 
C12 | Use adaptive transmission C24 | Utilize counter-drone techniques 


Table D.2 lists general mitigation categories into which specific counter measures can be 
classified, including: (D)etect, (N)eutralize, (L)imit, and (R)ecover as well as the 
corresponding effectiveness: (L)ow, (M)edium, (H)igh, and (V)ery high. Classification may 


depend on system and mission specific factors. 


Table D.2: Mitigation effectiveness notations 


39 


Effectiveness Mitigation Category [49] 
Detect Neutralize Limit Recover 
Very High DV NV LV RV 
High DH NH LH RH 
Medium DM NM LM RM 
Low DL NL LL RL 


Risk Assessment of selected UAS components utilized by MassDOT 


Standard practice (23) relies on fail-safe protocols for a UAS to execute scripted behavior such 
as return to base or hold in cases of jamming or disruption of communications. Hard-coded 
geofences are still vulnerable to override in the case of spoofing or signal hijacking. 
Omnidirectional antennas radiate in all directions to improve outdoor performance, but 
increase eavesdropping risk (18) on the commercial 2.4 and 5.8 GHz bands. 


The following discusses functionality and some potential vulnerabilities associated with UAS 
components utilized by MassDOT. However, a detailed mapping to attacks and potential 
countermeasures enumerated above is not performed here. 


DJI Inspire 2 is a platform integrating high definition video transmission (Security issues) 


In the past, the SSL Certificate for the DJI website has been compromised. DJI 
subsequently revoked this certificate and replaced it with a new certificate. Tampering 
with website content would have been possible. 

An independent security researcher reported that an Amazon Web Services server 
repository was accessible by unauthorized parties and was fixed in a day. Data could 
have been stolen or altered. 

The new Local Data Mode stops internet traffic to and from its DJI Pilot app, in order 
to enhance data privacy. However, this prevents connecting to the Internet and the DJI 
Pilot app cannot detect the user’s location, display the map and geofencing information 
such as No Fly Zones and temporary flight restrictions. Moreover, firmware updates 
will not be available. Telemetry data contained in flight logs such as altitude, distance, 
and speed will remain stored on the aircraft even if the user deactivates Local Data 
Mode, preventing utilization of real-time counter measures such as auto-pilot 
monitoring. 


AirMap provides access to airspace advisories, flight plan creation, and enables contact with 
airspace authorities. (Security issues) 


AirMap features include: an intrusion detection system, log analysis, and a web 
application firewall, penetration testing and vulnerability assessments, content delivery 
network, and data security controls. Compromise of these features can enable various 
attacks that could be executed throughout the mission lifecycle. 


Skyward is a drone-management platform 


Skyward’s interactive airspace allows viewing of flight restrictions, marking points of 
interest and hazards to be avoided. Syncing Skyward with DroneDeploy and DJI GO 
enables automated log upload, flight path visualization, and battery health monitoring. 
Compromise of Skyward can introduce vulnerabilities associated with its functionality, 
while compromise of Skyward while linked to DroneDeploy and DJI GO may enable 
introduction of vulnerabilities in to all aspects of functionality in DroneDeploy and DJI 
GO that interact with Skyward. 
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DroneDeploy is a cloud-based software to automate drone flight, capture data, and create maps 
and 3D models. (Security issues) 


While DroneDeploy implements many practices in support of data, web, and application 
security, compromise of these services can introduce corresponding vulnerabilities into the 
mission lifecycle. Assessment of data related services must consider the types of data 
transmitted through and stored on these platforms. Web and application security must consider 
the corresponding capabilities to identify and prioritize potential vulnerabilities. 


Data is encrypted in transit and at rest on DroneDeploy servers. 

Data is sent securely to DroneDeploy via the HTTPS protocol using the latest 
recommended ciphers and transparent LAN service (TLS). 

DroneDeploy is hosted on Amazon Web Services and Google Cloud. 

DroneDeploy employees do not have physical access to the Amazon or Google data 
centers, servers, network equipment, or storage, which can deter insider data theft. 
Annual network and system level penetration tests are conducted by an outside security 
vendor. 

Each software component undergoes a security risk assessment based on the Open Web 
Application Security Project (OWASP) Top 10, which is a list of the ten most critical 
security risks to web-based applications identified by consensus among web security 
organization. 

Application security includes password-based logins, and Google single sign-on for all 
accounts allowing use of Google or GSuite accounts to authenticate users requiring 
two-factor authentication with mechanisms including access codes or security keys. 
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Appendix E: Functional Decomposition and 
Data-flow 


A UAS may be decomposed into four functional modules (6) 
1. Navigation: Analyze sensory data for Autopilot decisions. 
2. Data Collection: Collect raw data for mission and UAS status. 
3. Communication: Send and receive the control signal and UAS data. 
4. Flight Control: Interact with other modules to preserve correct UAS flight state 


Navigation Module 


The navigation module is responsible for correctly stabilizing the UAS and navigating the UAS 
along a predefined path. Navigation can be performed automatically by the auto-pilot function 
of the drone or through manual control using a handheld controller over a wireless network 
setting. The auto-pilot function uses the predefined mission plan and sensing data generated 
from the navigational sensors noted above to navigate. In manual control, a user controls the 
navigation using line of sight communication with the UAS. Both the manual and auto-pilot 
modes are controlled by the central flight controller, which is primarily responsible for 
processing the sensor data. 


Figure E.1 shows the decomposition of a UAS into four functional modules (17) and data- 
flow. Navigation related functions and data flows are highlighted with bold arrows and boxes. 
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Figure E.1: UAS Functional Modules and Data-flow 
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Figure E.1 summarizes potential CAPEC attacks on functional modules of Navigation. 
Relevant CAPEC attacks include: forced deadlock, fault injection, and authentication bypass. 
For example, corruption of navigational data can lead to a forced deadlock. UAS functions 
such as environmental perception and the auto-pilot mechanism may be effected by this attack 
and the primary risks associated with this attack is failure of the autopilot software, resulting 
in a crash or fly-away. 


Table E.1: Functional modules and potential attacks on Navigation 


Navigation 
Potential Attacks Functional Modules Risk 
(checklist + literature) 
Attack Type 
(CAPEC) Attack ; 
Environmental . 
. Auto Pilot 
Perception 
Forced Deadlock | Corruption [23] Agee Apseied Autopilot —_ Error/Fail, 
Crash, Fly Away 
Fault Injecti Code Injection [18 
ee Sac jechon tte) Affected Affected Autopilot — Error/Fail, 
Crash, Fly Away 
False Data 
eee Autopilot — Error/Fail, 
Injection [18] Affected Affected Gash, Fly Away 
Authentication Rootkits [18] Affected Affected UAS Loss of Control, 
Bypass Crash 


Data Collection Module 


The data collection module acquires raw data during the mission and provides UAS mission 
control with necessary control data. The unit interacts with the environment to sense its 
surroundings. Sensor data may be divided into two types, navigational and mission specific. 
Navigational measurements such as magnetic sensors accelerometers, gyroscope sensors, and 
tilt sensors. These sensors provides an environmental perception about the UAS location, 
speed, position, stabilization, and orientation in real time. Possible mission specific sensors 
include cameras, infrared or night vision camera to take video and still images. Other type of 
mission specific sensors can be temperature and pressure sensors, which can provide additional 
information about the UAS operational environment. These sensor data are then sent to the 
navigational unit and flight control unit for further processing. 
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Figure E.2 shows data collection related functions and data flows highlighted with bold arrows 
and boxes. 
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Figure E.2: Functional modules and potential attacks on data collection 


44 


Table E.2 summarizes potential CAPEC attacks on functional modules of data collection. 


Table E.2: Functional modules and potential attacks on data collection 


Flight Control 
Risk 
Potential Attacks Hardware Component Funcnoyal . peceet 
Component literature) 
Attack Type 
(CAPEC) Attack Name ESCs Motors CFC 
Exploitation of 
Trusted Back Doors (18) Affected Autopilot Error/Fail 
Credentials 
Becta Firmware 
. Modification (18) | Affected Crash, Fly Away 
pees Supply Chain Crash, Fly Away 
Attack Attack (18) Affected 
Forced ; : ‘ 
Desaleek Corruption (23) Affected Autopilot Error/Fail 
Authentication . Autopilot 
nyo Rootkits (18) Affected Error/Fail 
Malicious Sleep Autopilot Error/Fail, 
Logic ces Affected 
: Deprivation (18) Crash 
Insertion 
Code Injection Autopilot Error/Fail, 
Fault (18) Affected Crash 
Injection False Data Affected Affected Autopilot Error/Fail, 
Injection (18) Crash 
Exploiting ; ; 
Trust in Fuzzing (18) Affected stanley Enron Fels 
; Crash 
Client 


Communication Module 


The Communication module is responsible for transmitting and receiving information, either 
from the user or from GPS satellites. There are three primary wireless communication links in 
a UAS. The first link uses a wireless network based on IEEE 802.11 2.4GHz channel. This 
channel is used for line of sight communication with the UAS. The second link is based on the 
cellular network to control the UAS remotely. The third communication link communicates 
with GPS satellites. 


Figure E.3 shows communication related functions and data flows highlighted with bold 
arrows and boxes. 
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Figure E.3: Communication module 
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Table E.3 summarizes potential CAPEC attacks on functional modules of communication. 


Table E.3: Functional modules and potential attacks on communication 


Communication 
Potential Attacks Functional Risk 
Components (checklist + literature) 
eta type Attack Name DataTx | GPS Tx 
(CAPEC) 
Input Data 
Manipulation Keyloggers (18) Affected Resource Leakage 
Authentication : Resource Leakage, 
Bypass Rootkits (18) Affected | Affected GCS failure 
Pepin Amst: | papi (ie) Affected GCS Failure 
in Client 
Sniffing (19) Affected | Affected | Resource Leakage 
Resource Leakage, 
Password : Se 
Cracking (18) Affected Automatic transmission 
Interception 6 Locked 
Eavesdropping (7) Affected | Affected | Resource Leakage 
oe Affected GCS Failure 
Distortion 
Reference Station ; 
Tpnemaenine Attack (18) Affected | Affected | GCS Failure 
Manipulation : GCS Failure, Automatic 
Signal Delay (18) Affected | Affected esac God Tucked 
Communication : : 
Channel Black Hole/Gray Hole (18) | Affected oo Hanne Sone 
a : Transmission Locked 
Manipulation 
Resource Leakage, 
Man-in-the-Middle (18) Affected Automatic Transmission 
Locked, GCS Failure 
Replay Attack (19) Affected | Affected | GCS Failure 
Address Resolution : 
Protocol (19) Affected GCS Failure 
; Resource Leakage, GCS 
Spoofing (19) Affected Failure 
Protocol Network Isolation (18) Affected GCS Failure 
Manipulation Rogue Node (18) Affected Resource Leakage, GCS 
Failure 
Hijacking (7) Affected oes DednaeesOCe 
Failure 
Cross Layer Attack (7) Affected mesource Deandue,GC> 
Failure 
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Resource Leakage, GCS 


Multi-Protocol Attack (7) Affected . 
Failure 
Traffic Injection Cmmd Injection (18) Affected GCS Failure 
J False Data Injection (18) Affected | Affected | GCS Failure 
: GCS Failure, 
Jamming (19) Affected | Affected Ticsscot Dara Thule 
Obstruction Deauthentication attack (19) | Affected | Affected cima 
Loss of Data Link 
GCS Failure, 
DoS (19) Affected | Affected Lesser ark 
Flight Control Module 


The flight control module consists of a central flight controller, which is responsible for 
processing the sensor and mission data, mission plan, and wireless communication data into 
electrical signals for the UAS motors and control circuits. The flight controller provides bi- 
directional communication between the data collection, communication, and navigation 


modules. 


Figure E.4 shows flight control related functions and data flows highlighted with bold arrows 


and boxes. 
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Figure E.4:; Functional modules and potential attacks on flight control 
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Table E.4 summarizes potential CAPEC attacks on functional modules of Flight Control. 


Table E.4: Flight control 


Flight Control 
Potential Attacks . clic - cee Risk 
omponen omponen (checklist 
Attack Type Attack ESCs Motors CFC : 7 
(CAPEC) Nanie literature) 
Exploitation ; 
of Trusted por Affected esa 
Credentials 
Firmware Craahi 
Modification Affected Fl Ms 
Hardware (18) pawey 
Integrity 
Attack Supply 
Chain Affected = au 
Attack (18) y y 
Forced Corruption Autopilot 
Deadlock (23) oe Error/Fail 
Authentication Rootkits Autopilot 
Bypass (18) neers Error/Fail 
Malicious Sleep Autopilot 
Logic Deprivation Affected Error/Fail, 
Insertion (18) Crash 
Code Autopilot 
Injection Affected Error/Fail, 
(18) Crash 
Fault Injection 
False Data Autopilot 
Injection Affected Affected Error/Fail, 
(18) Crash 
wi Autopilot 
Exploiine: Fuzzing (18) Affected Error/Fail, 
Trust in Client Gisch 
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Appendix F: Attacks Categorized According 
to Mission Stage and Category 


As noted in Figure 2.1, drone missions may be divided into pre-, in-, and post-flight. Attacks 
are classified into different categories according to the CAPEC standard developed by the 
MITRE Corporation (5) and links provided wherever possible. Our study was limited to the 
pre and inflight stages with emphasis on the inflight stage. Preflight categories considered 
include software and hardware, while inflight categories include software, hardware, 
communication, and physical security. 


Preflight 


Preflight software attacks 

Preflight software attacks include fault injection and authentication bypass. The following 
tables summarize these attacks, alternative names these attacks are known by, the physical 
component subject to the attack, mechanism of the attack according to the CAPEC, security 
service attribute affected, risk, current countermeasures taken by MassDOT (present in their 
checklist), and recommended techniques to mitigate the risk. 
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Table F.1: Preflight software attacks 


Attack type [CAPEC]: Fault Injection (624) 


Attack name 


Code Injections (18; 19), 


(Physical) component 


Firmware 


Mechanism of attack (CAPEC) 


Inject unexpected items into the code (152) 


Security service attribute affected Integrity 


Risk Crash Autopilot Software Error/Fail 
Current countermeasure activities Firmware updated. 
[MassDOT checklist] COTS related findings: AirMap has an Intrusion 


detection System [https://www.airmap.com/security/] 


Recommended mitigation 
techniques 


Regular patching of software (CAPEC) IDS (33) 


Attack type [CAPEC]: Authentication bypass (115) 


Mechanism of attack (CAPEC) 


Attack name Identity Spoofing (7) 
(Physical) component Software used, e.g, Skyward, AirMap, DroneDeploy 


Engage in Deceptive Interactions (156) 


Security service attribute affected 


Authentication, Confidentiality. 


Risk 


Hostile environment 


Current countermeasure activities 
[MassDOT checklist] 


Flight authorization; Flyaway in airport reviewed and 
informed to FAA; Map cached, Flight plan is built (setting 
altitudes, gimble angle). 


COTS related findings: 

DroneDeploy provides Google single sign-on for all 
accounts allowing use of Google or GSuite accounts to 
authenticate users requiring two-factor authentication. 
Google logins can be protected by multiple 2FA 
mechanisms including access codes or security keys. 
https://support.dronedeploy.com/docs/security-and- 


compliance. 


Recommended mitigation 
techniques 


Strong federated identity such as SAML to encrypt and 
sign identity tokens in transit. 

(Ref: CAPEC) 

Session timeout for all sessions. 

(Ref: CAPEC) 

Verify of authenticity of all 
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Table F.2: Preflight hardware attacks 


Attack type [CAPEC]: Malicious logic insertion (441) 


Attack name Sleep deprivation (2; 18) 

(Physical) component Battery 

Mechanism of attack (CAPEC) Manipulate System Resources (262) 
Security service attribute affected Integrity, Availability 

Risk Battery depletion 

Current countermeasure activities Batteries (controller, display) charged. 
[MassDOT checklist] Communication devices charged. 


COTS related findings: 

Syncing Skyward with Drone-Deploy and DJI GO 
enables visualize battery health. 
[https://community.skyward.io/s/dji-go-syncing- 
and-flight-visualizations] 

Recommended mitigation techniques Check for power leakage. 


Attack type [CAPEC]: Firmware Modification (638) 


Attack name Hardware Integrity Attack (6) 

(Physical) component ESC, Camera and other sensors 

Mechanism of attack (CAPEC) Manipulate System Resources (262) 

Security service attribute affected Integrity 

Risk Crash, Autopilot Software Error/Fail 

Current countermeasure activities UAS hardware inspected, registered and packed 
[MassDOT checklist] Rotors inspected, mounted. Camera fixed. Mission 


limitations and safety (eg. radio interference is 
checked, hazards/site Assessment) Weather check, 
Flyaway in airport reviewed and informed to FAA 
Recommended mitigation techniques Intrusion detection, (2) 

(normalcy profiling), CIDS (33) 

Penetration testing with the Attack tools 


Attack type [CAPEC]: Fault Injection (624) 


Attack name Fault Injection, Signals like EMP 
(electromagnetic pulses), laser pulses, clock glitches, 
etc.) (34) 
(Physical) component ESC, Barometer, Gyroscope, Accelerometer, Antenna 
Mechanism of attack (CAPEC) Inject unexpected items (152) 
Security service attribute affected Integrity, Availability 
Risk Crash, Autopilot Software Error/Fail 
Current countermeasure activities UAS hardware inspected, registered and packed 
[MassDOT checklist] Rotors inspected, mounted. Camera fixed. Mission 


limitations and safety (eg. radio interference is 
checked, hazards/site Assessment) Weather check, 
Flyaway in airport reviewed and informed to FAA 
Recommended mitigation techniques Intrusion detection, (2) 

(normalcy profiling), CIDS (33) Penetration testing 
with the Attack tools 
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In flight 


Inflight attacks are classified into four categories according to the CAPEC standard, namely 
software, hardware, communication, and physical security. 


Table F.3: Inflight software attacks 


Attack type [CAPEC]: Malicious logic Insertion (441) 


Attack name 


Spyware (23) 


(Physical) component 


GCS or Flight controller 


Mechanism of attack (CAPEC) 


Inject unexpected items (152) 


Security service attribute affected 


Integrity Confidentiality 


Risk 


Resource leak 


Current countermeasure activities 
[MassDOT checklist] 


Unknown 


Recommended mitigation techniques 


Attack name 


Anti—spyware software, firewalls, packet filters, 
Managing the security of the supply chain (22) 
Malwares (22) like Viruses/worms/Trojan/Rootkit/ 
keyloggers (18) 


(Physical) component 


GCS (18) Or Flight controller (on-board control unit) 
(19) 


Mechanism of attack (CAPEC) 
Security service attribute affected 


Inject unexpected items (152) 
Integrity 


Risk 


Current countermeasure activities 
[MassDOT checklist] 


Recommended mitigation techniques 


Crash 

Autopilot Software Error/Fail 

COTS related finding 

Air Map has a Web Application Firewall 

Ref: AirMap Website- 
https://www.airmap.com/security/ 
Anti-malware software, packet filters, firewalls, 
managing the security of the supply chain (22) 
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Table F.4: Inflight hardware attacks 


Attack type [CAPEC]: Protocol Manipulation (272) 


Attack name Sensor Spoofing (2; 18) 


(Physical) component GPS sensor, obstacle avoidance sensors, Camera 
sensor, other sensors like IR sensor, ultrasonic 
wave sensor, magnetometer, barometer (22) 


Mechanism of attack (CAPEC) Engage in Deceptive Interactions (156) 
Content spoofing (148) 

Security service attribute affected Integrity 

Risk Loss of GPS, crash, fly away (8; 19), auto pilot 
software error, loss of situational awareness 

Current countermeasure activities Manual monitoring of the flight checklist 

[MassDOT checkliist] activities for handling emergency conditions 


(mentioned in above table.) 


Recommended mitigation techniques Verify metadata along with actual data (35), 
cross verify data from redundant sensors (8), 
RANSAC algorithms 


Attack type [CAPEC]: Obstruction (607) 


Attack name Sensor Jamming (8) 


(Physical) component GPS sensor, obstacle avoidance sensors, camera 
sensor, other sensors like IR sensor, ultrasonic 
wave sensor, magnetometer 


Mechanism of attack (CAPEC) Manipulate System Resources (262) 
Security service attribute affected Integrity, Availability 
Risk Crash, fly away (8), auto pilot software error, 


loss of situational awareness 


Recommended mitigation techniques Measure the signal power level to detect 
jamming. (22), alternative navigation method 
[INS] (21)(Eg: Multiple camera system like 
MTS-B (8)) 


Attack type [CAPEC]: Interception (117) 


Attack name Sensor sniffing (19) 


(Physical) component GPS sensor, obstacle avoidance sensors, camera 
sensor, other sensors like IR sensor, ultrasonic 
wave sensor, magnetometer 


Mechanism of attack (CAPEC) Collect and analyze information 

Security service attribute affected Affected service attribute 

Risk Resource leak 

Recommended mitigation techniques Encryption (18) sensor firmware robustness 
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Table F.5: Inflight communications attacks 


Attack type [CAPEC]: Obstruction (607) 


Attack name Communication link Jamming (18) 
(Physical) component Control transmission and data transmission link 
Mechanism of attack Manipulate System Resources (262) 
(CAPEC) Obstruction (607) 
Security service attribute Availability 
affected 
Risk Loss of data link, Crash, Flyaway, Loss of direct visual 
Current countermeasure Manual monitoring of the flight. 
activities [MassDOT Handling Emergency situations based on 
checklist] UAS ability to execute scripted behavior (return to base or hold) (23) fail-safe 
protocol should be implemented. (23) 
1. Fly away 


a. Alert crew 
b. Press Home button 
c. Press kill if required 
2. Loss of datalink 
a. Alert crew 
b. Assess loss type 
c. Press Home Button 
3. Loss of GPS 
a. Wait in hover for 1 min to reconnect. 
b. Press Home Button 
4. Autopilot Software Error/Fail 
a. Stabilize aircraft. 
b. Switch to manual mode, fly towards GCS. 
5. Loss of Engine power 
a. Note the position of aircraft. 
b. Switch to manual mode. 
c. Fly to predetermined safe zone. 
d. If maintaining altitude, fly to home. 
6. GCS failure 
a. Home button pressed. 
b. Manual mode. 
c. Fly home. 
d. Land. 
7. Intrusion of Aircraft into UAS airspace 
a. Land immediately. 
8. Crash 
a. Switch to Manual mode. b. Safety procedures. 
Geo-fencing is used. 
COTS related findings: 
1) All data is sent securely to DroneDeploy via the HTTPS protocol using the 
latest recommended ciphers and TLS 
protocol.(Ref:https://support.dronedeploy.com/docs/security-and-compliance) 
2) DJI has recently launched a new Local Data Mode. This new mode stops 
internet traffic to and from its DJI Pilot app, in order to provide enhanced data 
privacy assurances. 


(Ref: https://www.expoUAS.com/news/latest/solution-cyber-vulnerabilities- 


dji-drones/) 
Recommended mitigation Adaptive Transmission, Channel switching, Backup channels Jamming 
techniques detection, Electronic Counter Measure techniques and jamming resistant 


modulations for security (19). 
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Attack type [CAPEC]: Traffic Injection (594) 


Attack name 


Command Injection (18; 36) 


(Physical) component 


Control transmission link 


Mechanism of attack (CAPEC) 


Inject unexpected items (152) 


Security service attribute affected Integrity 


Risk Crash, Autopilot Software Error/Fail 

Current countermeasure activities | See Obstruction under Inflight communications attacks 
[MassDOT checklist] above. 

Recommended mitigation User-controllable input should be validated and filtered for 
techniques potentially unwanted characters. (CAPEC) 


Whitelisting/blacklisting the inputs. (CAPEC) 
Location-based authentication (22)A lightweight Public 
Key Infrastructure (PKI). (22) 


Attack name 
(Physical) component 


False Data Injection (18; 22) 
Data transmission link 


Mechanism of attack (CAPEC) 


Security service attribute affected 


Engage in Deceptive Interactions (156) 
Content spoofing (148 
Integrity 


Risk 


Integrity 


Current countermeasure activities 


See Obstruction under Inflight communications attacks 


[MassDOT checklist] above. 
Recommended mitigation Fault detection approach (6) 
techniques Checking the meta data along with the data. (19) 


Attack type [CAPEC]: Exploiting Trust in Client (22) 


Attack name 


Fuzzing Attack (6; 18) 


(Physical) component 


Control transmission and data transmission link 


Mechanism of attack (CAPEC) 
Security service attribute affected 


Employ Probabilistic Techniques (223), fuzzing(28) 


Authentication 


Risk 


Illegal access of UAS which might lead to autopilot 
software error. 


Current countermeasure activities 
[MassDOT checklist] 
Recommended mitigation 
techniques 


See Obstruction under Inflight communications attacks 
above. 

Secure networking protocols. 

white-box and black-box fuzzing tests (6) 


Attack type [CAPEC]: Protocol Manipulation (272) 


Attack name 
(Physical) component 


Network Isolation (18) 
Control transmission and data transmission link 


Mechanism of attack (CAPEC) 


Manipulate System Resources (262) 
Infrastructure manipulation (161) 


Security service attribute affected 


Availability 


Risk 


Loss of data link (loss of communication) (19) 


Current countermeasure activities 


See Obstruction under Inflight communications attacks 


[MassDOT checklist] above 
Recommended mitigation Secure networking, Protocols, Redundant links (23) 
techniques 
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Attack type [CAPEC]: Communication Channel Manipulation (216) 


Attack name Black Hole/Gray Hole (18) 
(Physical) component Control transmission and data transmission link 
Mechanism of attack (CAPEC) Manipulate System Resources (262) 
Infrastructure manipulation (161) 
Security service attribute affected Integrity, Availability 
Risk Loss of communication, situational awareness, Crash 


Current countermeasure activities See Obstruction under Inflight communications attacks 
[MassDOT checklist] above 

Recommended mitigation Verifying the links, in fact, connected to the site they 
techniques intended, Secure network protocols. 


Attack type [CAPEC]: Interception (117) 


Attack name Packet Sniffing (18; 32) 

(Physical) component Data transmission link 

Mechanism of attack (CAPEC) Collect and Analyze Information (118) 
Security service attribute affected Confidentiality 

Risk Resource leak 


Current countermeasure activities See Obstruction under Inflight communications attacks 
[MassDOT checklist] above. 


Recommended mitigation Encryption (Combination of OTR and PGP (36) and 
techniques authentication [MAC] techniques. 
Attack name Password cracking (18) 
(Physical) component Control transmission and data transmission link 
Mechanism of attack (CAPEC) Employ Probabilistic Techniques (223) 
Brute force (112) 
Security service attribute affected Authentication, 
Confidentiality 
Risk Illegal access to UAS which might lead to resource leak, 


flyaway, crash 

Current countermeasure activities See Obstruction under Inflight communications attacks 
[MassDOT checklist] above. 

Recommended mitigation Strong passwords. (34) 

techniques 


Attack type [CAPEC]: Communication Channel Manipulation (216 
Attack name Man in the Middle attack (18; 22) 
(Physical) component Control transmission and data transmission link 
Mechanism of attack (CAPEC) Manipulate System Resources (262) 
Communication Channel Manipulation (216) 
Security service attribute affected Confidentiality, Integrity 
Risk Resource leak, Crash, Fly away, Loss of datalink ( 3" party 
can cause the link to disconnect) 
Current countermeasure activities See Obstruction under Inflight communications attacks 
[MassDOT checklist] above. 
Recommended mitigation Secure network protocols, Encryption (18), Authentication 
techniques Like One-time key (21) 
location-based authentication (22) 
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Attack type [CAPEC]: Obstruction (607) 


Attack name 


GPS signals jamming (18; 22) 


(Physical) component 


GPS signals jamming (18; 22) 


Mechanism of attack (CAPEC) 


Manipulate System Resources (262) 
Obstruction (607) 


Security service attribute affected 


Availability 


Risk 


Loss of GPS, Crash (19), Fly away, Auto Pilot Software 
error 


Current countermeasure activities 


See Obstruction under Inflight communications attacks 


[MassDOT checklist] above. 
Recommended mitigation Anti-Jamming techniques. 
techniques Jamming detection (22), Electronic 


Counter Measure techniques and jamming resistant 
modulations for security. (19) 


Attack type [CAPEC]: Authentication Bypass (115) 


Attack name 


De-authentication attack (19; 32) 


(Physical) component 


Control transmission and data transmission link 


Mechanism of attack (CAPEC) 
Security service attribute affected 


Manipulate System Resources (262) 
Flight safety, Availability 


Risk 


Loss of Data Link 
Drone can go to unexpected state (22), Ungraceful UAS 
operation shutdown (19; 22) 


Current countermeasure activities 


See Obstruction under Inflight communications attacks 


[MassDOT checklist] above. 
Recommended mitigation Strong authentication mechanisms and side channel 
techniques analysis (22) 


Use of Physical Unclonable functions (PUF) in 
authentication. (22) 


Attack type [CAPEC]: Protocol Manipulation (272) 


Attack name 


Rogue Node (18) 


Mechanism of attack (CAPEC) 


(Physical) component Control transmission and data transmission link 


Inject unexpected items (152) 


Security service attribute affected 


Confidentiality, Integrity, Availability 


Risk 


Hostile environment 


Current countermeasure activities 


See Obstruction under Inflight communications attacks 


[MassDOT checklist] above. 
Recommended mitigation Anomaly-based IDS (21) 
techniques 
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Table F.6: Inflight physical security attacks 
Attack type [CAPEC]: Physical Theft (507) 


Attack name Theft and Vandalism (22) 

(Physical) component Entire UAS or other physical components like camera 

Mechanism of attack (CAPEC) Subvert Access Control (225) 

Security service attribute affected Availability, Confidentiality 

Risk Loss or Damage to UAS/UAS components 

Current countermeasure activities Manual monitoring of the drone. 

[MassDOT checklist] 

Countermeasure Electronic immobilizer (22) alarms, and monitoring of 
targets. 


Attack type [CAPEC]: Obstruction (607) 


Attack name EMP or Laser pulses [CAPEC] 
(Physical) component ESC, Barometer, Gyroscope, Accelerometer, Antenna 
Mechanism of attack (CAPEC) Fault Injection (624) 


Security service attribute affected Availability 
Risk Crash 


Current countermeasure activities Manual monitoring of the drone. 
[MassDOT checklist] 

Countermeasure Sense and avoid features (22) 
Attack name Rogue Drone Collision Attack (20) 
(Physical) component Entire UAS in flight 
Mechanism of attack (CAPEC) Using rogue drones 

Security service attribute affected Availability 

Risk Crash 

Current countermeasure activities Manual monitoring of the drone. 
[MassDOT checklist] 

Countermeasure Counter-drone techniques. (20) 
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